-- Logs begin at Fri 2022-09-30 08:51:32 UTC, end at Fri 2022-09-30 10:08:22 UTC. -- Sep 30 09:36:56.376784 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 30 09:36:56.385275 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 30 09:36:56 2022] *** Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: nodename: dv-cinder-master-3-iscsi Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: machine: x86_64 Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: clock source: unix Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: pcre jit disabled Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: detected number of CPU cores: 4 Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: current working directory: / Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: detected binary path: /usr/bin/uwsgi-core Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: your processes number limit is 47680 Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: your memory page size is 4096 bytes Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: detected max file descriptor number: 2048 Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: lock engine: pthread robust mutexes Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: thunder lock: enabled Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 30 09:36:56.390938 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 30 09:36:56.426993 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: Python main interpreter initialized at 0x5569167b69c0 Sep 30 09:36:56.426993 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: python threads support enabled Sep 30 09:36:56.426993 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: your server socket listen backlog is limited to 100 connections Sep 30 09:36:56.426993 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: your mercy for graceful operations on workers is 90 seconds Sep 30 09:36:56.427328 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: mapped 403077 bytes (393 KB) for 2 cores Sep 30 09:36:56.427412 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: *** Operational MODE: preforking *** Sep 30 09:36:56.427701 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: *** uWSGI is running in multiple interpreter mode *** Sep 30 09:36:56.427701 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: spawned uWSGI master process (pid: 112416) Sep 30 09:36:56.427887 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 30 09:36:56.428450 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: spawned uWSGI worker 1 (pid: 112417, cores: 1) Sep 30 09:36:56.428785 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: spawned uWSGI worker 2 (pid: 112418, cores: 1) Sep 30 09:36:56.429480 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 30 09:36:57.081848 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 30 09:36:57.094026 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 30 09:36:58.159966 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:36:58.159966 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: from cryptography.utils import int_from_bytes Sep 30 09:36:58.160560 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:36:58.160560 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: from cryptography.utils import int_from_bytes Sep 30 09:36:58.162878 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:36:58.162878 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: from cryptography.utils import int_from_bytes Sep 30 09:36:58.163827 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:36:58.163827 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: from cryptography.utils import int_from_bytes Sep 30 09:37:00.638060 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 30 09:37:00.638652 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 30 09:37:00.639021 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 30 09:37:00.639121 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 30 09:37:00.639203 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 30 09:37:00.639292 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 30 09:37:00.639378 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 30 09:37:00.639457 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 30 09:37:00.639549 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 30 09:37:00.639698 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 30 09:37:00.639843 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.640078 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.640191 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.640400 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.640508 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.640711 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.640800 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.641027 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.641124 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.641331 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.641424 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.641644 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.641730 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.641958 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.642051 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.642274 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.642360 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.642533 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.642637 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.642870 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.642973 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.643183 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.643271 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.643497 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.643591 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.643795 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.643902 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.644101 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.644202 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.644409 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.644499 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.644704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.644804 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.645013 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.645109 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.645309 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.645404 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.645624 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.645797 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.646129 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.646228 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.646439 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.646565 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.646767 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.646930 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.647077 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.647192 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.647461 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.647627 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.647760 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.647940 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.648067 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.648258 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.648386 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.648563 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.648699 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.648880 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.649017 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.649182 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.649322 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.649582 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.649708 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.649893 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.650022 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.650198 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.650325 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.650506 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.650666 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.650951 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.651097 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.651258 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.651396 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.651560 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.651689 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.651878 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.652006 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.652194 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.652332 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.652501 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.652637 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.652822 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.652950 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.653136 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.653271 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.653449 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.653577 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.653763 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.653890 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.654061 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.654196 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.654360 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.654487 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.654684 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.654812 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.654999 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.655128 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.655299 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.655433 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.655597 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.655725 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.655874 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.656007 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.656136 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.656260 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.656432 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.656560 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.656743 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.656872 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.657058 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.657197 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.657324 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.657459 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.657586 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.657713 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.657838 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.657963 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.658133 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.658265 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.658434 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.658562 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.658759 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.658889 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.659099 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.659236 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.659363 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.659487 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.659620 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.659752 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.659919 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.660050 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.660216 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.660352 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.660514 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.660643 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.660826 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.660956 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.661131 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.661260 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.661446 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.661587 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.661746 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.661881 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.662059 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.662194 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.662450 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.39 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.662605 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.39 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.662735 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.39 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.662876 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.39 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.663098 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.663228 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.663352 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.663486 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.663654 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.663782 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.663957 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.664086 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.664270 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.664406 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.664582 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.664716 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.664883 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.665022 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.665182 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.665311 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.665480 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.665616 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.665795 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.665921 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.666120 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.666254 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.666417 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.666545 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.666721 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.666847 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.666989 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.667116 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.667252 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.667378 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.667547 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.667683 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.667847 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.667977 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.668146 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.668282 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.668443 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.668586 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.668719 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.668844 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.669022 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.669151 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.669319 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.669457 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.669617 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.669745 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.669915 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.670088 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.670215 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.670349 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.670499 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.670643 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.670824 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.670951 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.671122 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.671257 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.671419 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.671547 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.671716 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.671844 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.672028 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.672162 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.672344 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.672479 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.672644 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.672771 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.672943 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.673081 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.673238 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.673376 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.673534 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.673662 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.673846 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.673983 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.674142 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.674271 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.674441 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.674578 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.674780 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.674926 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.675078 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.675207 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.675377 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.675527 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.675690 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.675828 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.675987 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.676115 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.676283 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.676412 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.676595 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.676732 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.676887 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.677015 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.677184 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.677313 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.677495 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.677623 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.677789 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.677917 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.678085 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.678220 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.678380 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.678510 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.679206 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.679341 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.679505 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.679634 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.679801 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.679948 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.680096 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.680217 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.680390 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.680522 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.680684 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.680812 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.680995 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.681123 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.681310 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.681450 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.681607 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.681735 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.681905 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.682035 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.682202 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.682345 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.682514 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.682689 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:37:00.682871 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.682999 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.683197 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.683335 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.683534 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.683654 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.683840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.683970 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.684168 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.684299 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.684491 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.684619 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.684802 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.684931 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.685111 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.685239 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.685415 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.685542 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.685735 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.685862 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.686068 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.686203 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.686335 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.686462 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.686643 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.686793 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.686956 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.687107 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.687267 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.687405 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.687592 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.687719 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.687902 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.688046 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.688212 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.688348 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.688523 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.688660 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.688828 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.688964 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.689139 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.689272 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.689461 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.689600 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.689775 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.689904 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.690081 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.690207 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.690415 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.690555 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.690782 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.690940 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.691081 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.691207 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.691405 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.691540 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.691726 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.691853 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.692038 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.692172 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.692348 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.692477 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.692660 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.692795 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.692999 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.693127 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.693306 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.693434 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.693610 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.693736 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.693921 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.694043 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.694249 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.694913 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.695135 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.695264 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.695451 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.695590 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.695764 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.695901 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.696070 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.696196 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.696375 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.696503 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.696680 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.696810 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.697012 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.697147 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.697321 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.697457 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.697639 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.697766 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.697948 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.698075 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.698254 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.698392 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.698566 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.698734 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.698894 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.699035 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.699199 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.699342 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.699503 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.699632 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.699802 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.699939 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.700101 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.700241 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.700398 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.700527 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.700700 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.700837 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.700999 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.701128 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.701313 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.701441 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.701629 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.701764 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.701929 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.702066 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.702254 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.702384 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.702554 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.702721 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.702884 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.703013 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.703193 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.703320 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.703497 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.703625 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.703795 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.703936 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.704099 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.704234 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.704402 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.704538 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.704704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.704833 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.705022 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.705148 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.705327 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.705463 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.705628 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.705766 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.705942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.706079 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.706242 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.706385 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.706542 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.706717 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.706876 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.707006 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.707182 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.707338 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.707474 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.707607 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.707779 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.707907 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.708084 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.708222 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.708400 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.708530 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.708705 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.708834 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.709035 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.709173 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.709305 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.709432 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.709632 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.709761 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.709958 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.710086 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.710280 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.710407 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.710613 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.710745 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.710933 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.711074 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.711253 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.711391 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.711519 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.711653 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.711803 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.711932 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.712109 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.712239 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.712398 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.712537 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.712700 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.712829 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.712998 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.713125 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.713292 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.713429 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.713589 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.713719 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.713886 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.714006 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.714181 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.714317 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.714498 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.714648 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.714824 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.714963 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.715123 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.715253 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.715419 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.715557 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.715716 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.715844 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.716052 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.716181 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.716350 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.716486 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.716683 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.716812 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.717018 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.717155 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.717320 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.717447 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.717630 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.717758 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.717934 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.718062 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.718242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.718370 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.718545 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.718743 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.718873 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.718999 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.719167 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.719296 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.719470 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.719609 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.719784 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.719911 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.720086 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.720221 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.720398 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.720535 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.720701 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.720829 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.721019 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.721147 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.721320 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.721460 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.721633 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.721770 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.721933 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.722073 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.722239 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.722376 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.722574 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.722726 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.722925 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.723061 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.723234 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.723362 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.723551 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.723686 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.723854 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.723981 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.724161 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.724290 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.724466 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.724594 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.724772 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.724899 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.725074 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.725209 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.725376 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.725512 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.725711 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.725840 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.726014 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.726151 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.726325 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.726453 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.726647 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.726779 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.726967 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.727107 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.727272 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.727401 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.727584 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.727722 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.727882 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.728012 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.728189 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.728318 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.728511 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.728646 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.728836 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.728972 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.729140 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.729269 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.729446 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.729582 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.729759 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.729895 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.730065 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.730202 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.730367 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.730510 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.730694 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.730837 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.730996 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.731125 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.731295 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.731424 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.731598 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.731727 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.731906 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.732037 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.732210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.732341 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.732513 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.732642 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.732812 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.732939 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.733125 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.733265 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.733428 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.733556 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.733727 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.733854 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.734024 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.734152 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.734330 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.734449 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.734679 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.734818 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.735183 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.735722 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.735722 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.735966 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.735966 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.736212 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.736348 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.736476 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.736601 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.736733 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.736868 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.737003 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.737130 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.737268 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.737395 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.737522 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.737653 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.737857 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.737986 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.738111 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.738236 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.738369 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.738505 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.738693 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.738873 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.739017 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.739147 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.739285 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.739477 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.739609 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.739790 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.739926 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.740101 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.740229 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.740412 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.740541 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.740725 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.740853 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.741047 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.741727 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.741875 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.742003 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.742170 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.742312 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.742482 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.742658 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.742843 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.742985 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.743158 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.743287 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.743470 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.743609 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.743795 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.743930 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.744103 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.744244 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.744461 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.744603 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.744792 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.744920 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.745096 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.745247 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.745436 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.745576 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.745748 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.745886 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.746359 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.746509 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.746795 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.746941 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.747079 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.747205 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.747375 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.747504 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.747720 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.747916 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.748089 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.748236 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.748386 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.748516 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.748714 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.748844 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.749027 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.749178 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.749362 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.749504 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.749679 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.749818 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.750021 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.750161 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.750288 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.750433 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.750646 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.750779 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.750957 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.751150 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.751304 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.751451 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.751602 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.751742 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.751882 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.752010 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.752174 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.752315 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.752481 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.752628 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.752817 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.752956 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.753123 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.753257 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.753435 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.753574 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.753739 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.753868 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.754050 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.754182 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.754355 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.754485 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.754701 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.754846 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.755015 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.755144 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.755334 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.755528 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.755672 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.755810 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.755951 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.756122 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.756274 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.756764 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.757410 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.757639 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.757761 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.757990 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.758199 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.758402 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.758534 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.758733 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.758875 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.759043 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.759182 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.759353 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.759483 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.759667 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.759798 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.759973 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.760132 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.760262 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.760408 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.760611 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.760740 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.760923 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.761054 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.761229 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.761358 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.761532 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.761669 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.761841 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.762069 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.762198 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.762322 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.762457 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.762627 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.762793 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.762924 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.763105 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.763245 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.763422 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.763551 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.763753 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.763882 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.764008 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.764133 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.764310 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.764451 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.764628 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.764758 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.764939 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.765076 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.765244 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.765385 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.765547 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.765675 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.765844 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.765972 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.766150 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.766285 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.766446 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.766575 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.766785 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.766916 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.767088 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.767229 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.767861 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.767991 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.767991 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.768214 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.768340 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.768475 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.768603 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.768747 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.768880 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.769005 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.769140 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.769276 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.769411 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.769537 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.769670 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.769796 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.769939 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.770066 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.770191 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.770317 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.770450 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.770576 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.770749 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.770885 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.771021 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.771137 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.771260 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.771385 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.771524 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.771659 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.771833 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.771964 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.772135 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.772273 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.772460 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.772591 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.772762 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.772897 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.773060 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.773202 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.773360 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.773474 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.773663 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.773791 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.773959 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.774095 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.774257 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.774387 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.774557 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.774727 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.774879 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.775024 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.775199 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.775326 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.775503 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.775631 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.775804 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.775934 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.776102 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.776231 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.776398 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.776525 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.776737 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.776879 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.777037 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.777166 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.777382 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.777508 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.777685 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.777812 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.777996 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.778125 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.778305 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.778440 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.778631 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.778763 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.778944 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.779086 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.779247 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.779385 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.779570 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.779699 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.779881 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.780009 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.780193 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.780331 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.780497 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.780625 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.780803 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.780933 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.781124 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.781254 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.781435 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.781564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.781739 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.781873 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.782051 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.782195 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.782369 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.782496 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.782711 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.782839 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.783017 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.783154 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.783319 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.783453 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.783625 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.783752 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.783930 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.784057 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.784235 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.784362 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.784541 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.784668 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.784847 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.784976 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.785152 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.785288 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.785458 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.785585 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.785760 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.785888 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.786065 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.786194 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.786366 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.786492 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.786694 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.786825 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.786999 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.787137 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.787300 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.787441 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.787599 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.787727 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.787917 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.788158 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.788293 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.788426 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.788561 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.788687 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.788822 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.788948 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.789104 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.789243 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.789407 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.789536 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.789706 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.789836 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.790125 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.790253 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.790428 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.790563 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.790755 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.790885 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.791051 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.791179 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.791355 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.791484 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.791659 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.791795 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.792005 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.792135 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.792261 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.792397 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.792621 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.792748 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.792955 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.793084 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.793264 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.793394 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.793574 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.793703 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.793879 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.794009 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.794181 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.794310 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.794488 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.794653 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.794820 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.794950 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.795126 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.795253 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.795429 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.795575 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.795725 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.795860 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.796024 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.796155 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.796331 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.796512 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.796642 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.796766 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.796928 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.797057 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.797338 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.797476 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.797637 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.797771 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.797935 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.798061 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.798344 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.798474 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.798671 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.798802 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.798977 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.799118 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.799296 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.799435 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.799613 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.799750 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.799931 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.800061 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.800232 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.800362 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.800541 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.800674 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.800844 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.800981 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.801161 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.801290 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.801467 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.801595 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.801770 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.801905 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.802070 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.802209 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.802412 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.802540 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.802714 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.802842 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.803002 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.803150 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.803312 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.803441 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.804157 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.804293 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.804554 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.804692 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.804865 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.804986 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.805168 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.805325 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.805484 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.805626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.805786 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.805912 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.806087 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.806216 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.806386 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.806514 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.806709 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.806839 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.807010 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.807136 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.807310 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.807441 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.807620 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.807746 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.807919 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.808046 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.808225 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.808353 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.808527 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.808653 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.808828 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.808954 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.809128 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.809256 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.809427 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.809557 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.809733 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.809868 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.810031 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.810159 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.810349 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.810480 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.810680 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.810832 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.811002 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.811136 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.811306 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.811434 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.811605 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.811742 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.811921 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.812048 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.812221 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.812348 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.812520 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.812656 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.812817 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.812946 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.813116 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.813244 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.813444 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.813583 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.813744 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.813871 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.814063 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.814192 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.814388 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.814516 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.814721 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.814851 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.815033 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.815177 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.815361 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.815490 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.815678 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.815805 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.815985 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.816124 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.816289 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.816417 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.816601 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.816731 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.816900 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.817028 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.817200 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.817328 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.817496 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.817630 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.817798 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.817933 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.818095 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.818223 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.818395 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.818529 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.818700 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.818843 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.819010 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.819146 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.819833 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.819967 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.820150 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.820290 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.820463 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.820591 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.820763 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.820898 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.821070 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.821198 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.821379 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.821506 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.821683 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.821811 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.821982 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.822118 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.822280 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.822409 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.822578 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.822730 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.822929 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.823058 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.823227 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.823356 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.823550 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.823685 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.823858 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.823990 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.824173 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.824300 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.824484 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.824621 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.824795 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.824923 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.825102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.825237 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.825412 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.825552 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.825723 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.825851 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.826032 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.826160 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.826339 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.826475 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.826665 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.826795 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.826981 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.827111 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.827291 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.827429 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.827604 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.827806 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.828120 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.828250 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.828434 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.828571 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.828743 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.828870 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.829054 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.829182 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.829361 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.829489 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.829669 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.829797 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.829975 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.830103 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.830285 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.830415 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.830665 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.830797 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.830934 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.831060 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.831218 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.831349 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.831541 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.831671 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.831858 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.832060 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.832190 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.832324 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.832494 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.832631 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.832805 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.832944 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.833115 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.833245 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.833423 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.833559 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.833731 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.833862 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.834071 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.834200 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.834376 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.834506 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.834709 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.834858 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.835045 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.835184 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.835355 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.835485 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.835663 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.835791 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.835967 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.836095 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.836272 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.836411 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.836580 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.836710 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.836885 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.837012 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.837192 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.837322 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.837494 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.837622 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.837802 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.837931 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.838108 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.838237 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.838427 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.838557 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.838776 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.838917 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.839089 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.839220 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.839395 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.839531 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.839705 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.839835 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.840015 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.840225 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.840529 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.840659 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.840838 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.840976 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.841166 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.841305 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.841484 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.841611 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.841800 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.841938 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.842108 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.842244 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.842419 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.842561 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.842745 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.842887 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.843049 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.843178 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.843346 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.843475 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.843653 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.843792 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.843954 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.844081 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.844251 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.844381 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.844568 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.844706 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.844865 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.845000 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.845163 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.845300 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.845476 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.845599 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.845779 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.846003 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.846301 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.846435 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.846658 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.846793 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.846970 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.847100 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.847291 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.847421 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.847604 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.847734 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.847910 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.848039 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.848210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.848347 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.848527 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.848657 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.848826 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.848962 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.849126 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.849265 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.849433 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.849570 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.849736 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.849864 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.850034 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.850181 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.850334 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.850985 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.851173 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.851304 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.851481 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.851786 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.852121 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.852250 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.852423 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.852554 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.852743 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.852882 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.853045 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.853174 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.853344 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.853474 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.853642 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.853774 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.853949 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.854086 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.854248 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.854386 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.854553 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.854728 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.854879 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.855009 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.855177 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.855307 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.855481 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.855619 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.855777 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.855906 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.856082 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.856219 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.856381 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.856511 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.856682 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.856809 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.856994 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.857126 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.857318 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.857446 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.857617 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.857747 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.857937 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.858074 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.858235 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.858363 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.858534 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.858682 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.858873 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.859014 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.859177 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.859305 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.859488 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.859622 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.859790 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.859920 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.860094 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.860232 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.860406 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.860534 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.860734 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.860863 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.861054 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.861259 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.861589 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.861716 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.861922 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.862041 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.862217 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.862347 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.862546 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.862740 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.862865 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.862984 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.863163 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.863465 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.863578 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.863785 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.864091 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.864405 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.864708 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.864925 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.865054 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.865247 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.865378 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.865580 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.865710 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.865890 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.866018 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.866776 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.866909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.867094 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.867227 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.867404 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.867543 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.867710 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.867841 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.868016 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.868146 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.868322 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.868460 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.868628 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.868765 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.868937 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.869066 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.869249 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.869388 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.869560 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.869690 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.869870 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.869999 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.870180 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.870321 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.870490 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.870648 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.870820 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.870950 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.871126 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.871271 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.871431 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.871561 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.871735 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.871865 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.872044 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.872166 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.872369 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.872509 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.872676 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.872805 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.872975 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.873104 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.873281 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.873589 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.873718 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.873903 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.874040 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.874234 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.874364 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.874549 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.874712 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.874881 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.875012 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.875201 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.875521 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.875649 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.875831 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.875970 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.876150 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.876279 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.876460 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.876598 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.876774 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.876905 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.877102 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.877232 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.877428 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.877556 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.877740 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.877876 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.878042 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.878177 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.878344 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.878473 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.878667 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.878800 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.878976 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.879107 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.879290 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.879428 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.879596 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.879734 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.879895 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.880026 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.880195 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.880333 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.880500 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.880623 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.880799 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.880929 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.881099 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.881239 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.881662 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.881820 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.881984 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.882115 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.882287 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.882425 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.882608 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.882741 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.882907 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.883044 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.883207 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.883346 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.883506 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.883637 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.883803 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.883932 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.884099 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.884231 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.884400 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.884528 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.884702 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.884829 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.885000 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.885131 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.885300 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.885436 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.885598 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.885725 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.885895 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.886017 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.886202 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.886331 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.886506 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.886691 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.886821 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.886946 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.887123 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.887252 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.887427 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.887570 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.887732 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.887871 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.888048 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.888218 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.888349 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.888486 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.888667 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.888796 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.888979 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.889109 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.889286 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.889415 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.889607 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.889745 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.889912 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.890040 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.890219 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.890350 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.890526 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.890673 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.890864 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.890991 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.891170 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.891299 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.891475 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.891606 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.891796 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.891926 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.892124 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.892251 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.892402 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.892537 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.892711 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.892856 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.893021 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.893148 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.893328 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.893456 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.893657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.893795 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.893985 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.894114 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.894292 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.894420 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.894641 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.894787 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.894949 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.895050 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.895254 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.895360 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.895561 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.895659 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.895865 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.895962 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.896175 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.896309 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.896482 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.896613 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.896786 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.896916 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.897094 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.897223 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.897426 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.897549 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.897734 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.897863 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.898051 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.898193 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.898356 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.898491 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.898689 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.898818 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.899009 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.899156 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.899325 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.899462 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.899642 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.899769 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.899965 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.900102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.900277 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.900406 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.900591 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.900719 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.900898 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.901038 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.901197 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.901326 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.901452 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 30 09:37:00.901577 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.901878 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.902206 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.902521 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.902860 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.903166 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:37:00.903409 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 30 09:37:01.053697 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO dbcounter [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Registered counter for database nova_api Sep 30 09:37:01.057092 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO dbcounter [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Registered counter for database nova_api Sep 30 09:37:01.075644 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_db.sqlalchemy.engines [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112418) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:37:01.078965 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_db.sqlalchemy.engines [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:37:01.082374 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writer thread running {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:37:01.085422 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writer thread running {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:37:01.139459 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.context [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),f6355185-7b23-453c-870b-129ec40cec19(cell1) {{(pid=112418) load_cells /opt/stack/nova/nova/context.py:464}} Sep 30 09:37:01.141698 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.context [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),f6355185-7b23-453c-870b-129ec40cec19(cell1) {{(pid=112417) load_cells /opt/stack/nova/nova/context.py:464}} Sep 30 09:37:01.143273 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:37:01.143753 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:37:01.145306 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:37:01.145573 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_concurrency.lockutils [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:37:01.146035 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:37:01.146125 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_concurrency.lockutils [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:37:01.146487 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:37:01.147661 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_concurrency.lockutils [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:37:01.148383 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_concurrency.lockutils [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:37:01.148471 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:37:01.148829 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_concurrency.lockutils [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:37:01.150443 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO dbcounter [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Registered counter for database nova_cell0 Sep 30 09:37:01.150814 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_concurrency.lockutils [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:37:01.152787 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO dbcounter [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Registered counter for database nova_cell0 Sep 30 09:37:01.154507 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO dbcounter [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Registered counter for database nova_cell1 Sep 30 09:37:01.156716 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO dbcounter [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Registered counter for database nova_cell1 Sep 30 09:37:01.161691 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_db.sqlalchemy.engines [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112418) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:37:01.164420 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_db.sqlalchemy.engines [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112418) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:37:01.165612 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_db.sqlalchemy.engines [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:37:01.167492 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_db.sqlalchemy.engines [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:37:01.167842 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writer thread running {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:37:01.172895 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writer thread running {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:37:01.175429 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writer thread running {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:37:01.311723 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writer thread running {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:37:01.320407 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO dbcounter [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Registered counter for database nova_cell0 Sep 30 09:37:01.356701 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_db.sqlalchemy.engines [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112418) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:37:01.356874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO dbcounter [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Registered counter for database nova_cell0 Sep 30 09:37:01.357001 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writer thread running {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:37:01.357090 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG oslo_db.sqlalchemy.engines [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:37:01.357090 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writer thread running {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:37:01.748681 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: WARNING keystonemiddleware.auth_token [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 30 09:37:01.755850 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: WARNING keystonemiddleware.auth_token [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 30 09:37:01.769455 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: WARNING keystonemiddleware.auth_token [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 30 09:37:01.776651 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: WARNING keystonemiddleware.auth_token [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 30 09:37:02.101387 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: WARNING keystonemiddleware.auth_token [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 30 09:37:02.107263 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: WARNING keystonemiddleware.auth_token [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 30 09:37:02.112502 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5569167b69c0 pid: 112418 (default app) Sep 30 09:37:02.114442 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: WARNING keystonemiddleware.auth_token [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 30 09:37:02.114551 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:37:02.115720 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] 10.222.0.39 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001707 Sep 30 09:37:02.116664 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 788 bytes} [Fri Sep 30 09:37:02 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:37:02.121056 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: WARNING keystonemiddleware.auth_token [None req-1d18f832-95e1-4c27-849f-9b08d4aef8f8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 30 09:37:02.126278 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5569167b69c0 pid: 112417 (default app) Sep 30 09:38:25.093030 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:25.094037 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001460 Sep 30 09:38:25.094629 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 2/2] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:25.106296 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writing DB stats nova_api:SELECT=3 {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:25.107398 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writing DB stats nova_cell0:SELECT=3 {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:25.108246 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writing DB stats nova_cell1:SELECT=3 {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:25.109346 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:25.979832 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-f9d78705-ef8a-4c95-be33-59c61dc2fe7f admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:26.012532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-f9d78705-ef8a-4c95-be33-59c61dc2fe7f admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.911263 Sep 30 09:38:26.013275 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 1/3] 10.222.0.39 () {60 vars in 1292 bytes} [Fri Sep 30 09:38:25 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 913 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:28.670129 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:28.671193 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-da4585d2-7ecf-4507-9669-791549c152d6 None None] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001459 Sep 30 09:38:28.671752 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 3/4] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:29.110273 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-38a73725-3df6-4d3c-bb3a-c9cdfa429bd4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "cirros256", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "disk": 1, "vcpus": 1, "swap": 0, "ram": 256, "id": "c1", "rxtx_factor": 1.0}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:29.208375 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-38a73725-3df6-4d3c-bb3a-c9cdfa429bd4 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.530679 Sep 30 09:38:29.208950 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 2/5] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:28 2022] POST /compute/v2.1/flavors => generated 407 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:38:29.223573 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_api:SELECT=3 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:29.224352 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_cell1:SELECT=3 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:29.226361 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_cell0:SELECT=3 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:29.227811 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:29.438528 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-2bc91336-cc4c-4bf0-90af-b7665e9fd0b3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:29.559887 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-2bc91336-cc4c-4bf0-90af-b7665e9fd0b3 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.342093 Sep 30 09:38:29.560471 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 4/6] 10.222.0.39 () {64 vars in 1336 bytes} [Fri Sep 30 09:38:29 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:32.172862 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-38a73725-3df6-4d3c-bb3a-c9cdfa429bd4 admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:32.173927 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-38a73725-3df6-4d3c-bb3a-c9cdfa429bd4 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Sep 30 09:38:32.175411 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 3/7] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:32 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:32.823847 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-c41ee7a5-f408-46b6-9eaa-d4a15129c390 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d1", "vcpus": 1, "os-flavor-access:is_public": true, "name": "ds512M", "swap": 0, "disk": 5}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:32.964870 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-c41ee7a5-f408-46b6-9eaa-d4a15129c390 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.782964 Sep 30 09:38:32.965605 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 5/8] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:32 2022] POST /compute/v2.1/flavors => generated 404 bytes in 784 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 30 09:38:32.978292 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-9b61f404-4da5-4331-97c8-ed7dbbe3c2b5 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:33.076493 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-9b61f404-4da5-4331-97c8-ed7dbbe3c2b5 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.101952 Sep 30 09:38:33.077262 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 4/9] 10.222.0.39 () {64 vars in 1336 bytes} [Fri Sep 30 09:38:32 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:35.658982 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-c41ee7a5-f408-46b6-9eaa-d4a15129c390 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:35.660156 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-c41ee7a5-f408-46b6-9eaa-d4a15129c390 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001594 Sep 30 09:38:35.660745 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 6/10] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:36.076717 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-bc254784-66ca-4e1e-8a46-53279ba2d2e8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d2", "disk": 10, "vcpus": 1, "name": "ds1G", "ram": 1024, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:36.163812 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-bc254784-66ca-4e1e-8a46-53279ba2d2e8 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.496888 Sep 30 09:38:36.164365 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 5/11] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:35 2022] POST /compute/v2.1/flavors => generated 404 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:38:36.176824 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-6fcbdeb7-29b5-40a4-a178-aaca02601feb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:36.271374 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-6fcbdeb7-29b5-40a4-a178-aaca02601feb admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.098708 Sep 30 09:38:36.271951 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 7/12] 10.222.0.39 () {64 vars in 1336 bytes} [Fri Sep 30 09:38:36 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:38.876322 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-bc254784-66ca-4e1e-8a46-53279ba2d2e8 admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:38.877271 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-bc254784-66ca-4e1e-8a46-53279ba2d2e8 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001475 Sep 30 09:38:38.877728 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 6/13] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:39.295844 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-b288887b-daac-4680-8694-a7205b2da2c0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "ram": 2048, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds2G", "vcpus": 2, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "disk": 10}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:39.369690 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-b288887b-daac-4680-8694-a7205b2da2c0 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.484947 Sep 30 09:38:39.370515 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 8/14] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:38 2022] POST /compute/v2.1/flavors => generated 404 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:38:39.383261 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-837cc8ce-faff-496e-9b19-76b6393abb72 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:39.476752 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-837cc8ce-faff-496e-9b19-76b6393abb72 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.097543 Sep 30 09:38:39.477675 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 7/15] 10.222.0.39 () {64 vars in 1336 bytes} [Fri Sep 30 09:38:39 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:42.123806 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-b288887b-daac-4680-8694-a7205b2da2c0 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:42.124958 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-b288887b-daac-4680-8694-a7205b2da2c0 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001964 Sep 30 09:38:42.125465 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 9/16] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:42 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:42.546965 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-53d25814-0c01-43e7-9f90-972b6ee2ce5d admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "id": "d4", "rxtx_factor": 1.0, "name": "ds4G", "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 20, "os-flavor-access:is_public": true, "ram": 4096}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:42.602999 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-53d25814-0c01-43e7-9f90-972b6ee2ce5d admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.468966 Sep 30 09:38:42.603561 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 8/17] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:42 2022] POST /compute/v2.1/flavors => generated 404 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:38:42.617686 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-8154c42c-d456-4515-b8eb-a6faf8186342 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:42.693934 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-8154c42c-d456-4515-b8eb-a6faf8186342 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.080404 Sep 30 09:38:42.694846 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 10/18] 10.222.0.39 () {64 vars in 1336 bytes} [Fri Sep 30 09:38:42 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:45.434346 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-53d25814-0c01-43e7-9f90-972b6ee2ce5d admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:45.435465 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-53d25814-0c01-43e7-9f90-972b6ee2ce5d admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001601 Sep 30 09:38:45.435872 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 9/19] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:45.869538 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-4b47dbc6-1113-4518-aceb-929be7bb3bd3 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:45.895129 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-4b47dbc6-1113-4518-aceb-929be7bb3bd3 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.452536 Sep 30 09:38:45.895668 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 11/20] 10.222.0.39 () {60 vars in 1292 bytes} [Fri Sep 30 09:38:45 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:38:48.477146 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-53d25814-0c01-43e7-9f90-972b6ee2ce5d admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:48.478257 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-53d25814-0c01-43e7-9f90-972b6ee2ce5d admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001549 Sep 30 09:38:48.478726 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 10/21] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:48 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:48.888410 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-a2b7a794-b38a-4a99-94d3-c90ce26114d0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "rxtx_factor": 1.0, "ram": 512, "id": "1", "name": "m1.tiny", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "swap": 0, "os-flavor-access:is_public": true}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:48.956515 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-a2b7a794-b38a-4a99-94d3-c90ce26114d0 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.471635 Sep 30 09:38:48.957688 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 12/22] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:48 2022] POST /compute/v2.1/flavors => generated 402 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 30 09:38:48.969523 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-e6d7912b-ba0b-4b4e-8b55-dd6596c8805d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:49.051828 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-e6d7912b-ba0b-4b4e-8b55-dd6596c8805d admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.085779 Sep 30 09:38:49.052345 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 11/23] 10.222.0.39 () {64 vars in 1333 bytes} [Fri Sep 30 09:38:48 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:51.630550 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-a2b7a794-b38a-4a99-94d3-c90ce26114d0 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:51.631845 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-a2b7a794-b38a-4a99-94d3-c90ce26114d0 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001741 Sep 30 09:38:51.632239 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 13/24] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:51 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:52.094289 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-6f9c7cb7-3222-409e-8104-5507c361b93c admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "2", "rxtx_factor": 1.0, "swap": 0, "name": "m1.small", "ram": 2048, "os-flavor-access:is_public": true, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:52.162320 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-6f9c7cb7-3222-409e-8104-5507c361b93c admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.523842 Sep 30 09:38:52.162890 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 12/25] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:51 2022] POST /compute/v2.1/flavors => generated 405 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:38:52.174568 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-2d802493-7baa-4f70-a395-4d99fae25b1e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:52.269322 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-2d802493-7baa-4f70-a395-4d99fae25b1e admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.097744 Sep 30 09:38:52.269906 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 14/26] 10.222.0.39 () {64 vars in 1333 bytes} [Fri Sep 30 09:38:52 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:55.004973 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-6f9c7cb7-3222-409e-8104-5507c361b93c admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:55.006260 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-6f9c7cb7-3222-409e-8104-5507c361b93c admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001503 Sep 30 09:38:55.006966 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 13/27] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:55 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:55.470857 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-3259e215-8a42-401b-9bd0-bd41d929e14c admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "disk": 40, "ram": 4096, "name": "m1.medium", "os-flavor-access:is_public": true, "swap": 0, "rxtx_factor": 1.0, "vcpus": 2, "id": "3"}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:55.551953 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-3259e215-8a42-401b-9bd0-bd41d929e14c admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.539273 Sep 30 09:38:55.552789 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 15/28] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:55 2022] POST /compute/v2.1/flavors => generated 406 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:38:55.564717 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-9514d515-4847-4ba4-aa12-9676c82e6678 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:55.571634 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:55.646901 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-9514d515-4847-4ba4-aa12-9676c82e6678 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.085562 Sep 30 09:38:55.647907 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 14/29] 10.222.0.39 () {64 vars in 1333 bytes} [Fri Sep 30 09:38:55 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:38:58.255574 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-3259e215-8a42-401b-9bd0-bd41d929e14c admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:38:58.257894 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-3259e215-8a42-401b-9bd0-bd41d929e14c admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002742 Sep 30 09:38:58.258320 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 16/30] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:38:58 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:38:58.696376 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-91585adb-c1b1-44d4-8a2f-1ca78cf42936 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 80, "name": "m1.large", "vcpus": 4, "rxtx_factor": 1.0, "ram": 8192, "id": "4", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:58.763884 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-91585adb-c1b1-44d4-8a2f-1ca78cf42936 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.499415 Sep 30 09:38:58.770657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 15/31] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:38:58 2022] POST /compute/v2.1/flavors => generated 405 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:38:58.776692 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-4194a340-1cfb-45c5-9982-d225dd386d23 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:38:58.840396 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-4194a340-1cfb-45c5-9982-d225dd386d23 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.067334 Sep 30 09:38:58.841441 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 17/32] 10.222.0.39 () {64 vars in 1333 bytes} [Fri Sep 30 09:38:58 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:39:01.457614 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-91585adb-c1b1-44d4-8a2f-1ca78cf42936 admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:01.458548 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-91585adb-c1b1-44d4-8a2f-1ca78cf42936 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Sep 30 09:39:01.459136 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 16/33] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:39:01 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:01.903602 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-a1cb1d6a-b47d-4e0d-92e8-f8b82f1f11ad admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 160, "vcpus": 8, "os-flavor-access:is_public": true, "id": "5", "rxtx_factor": 1.0, "swap": 0, "ram": 16384, "name": "m1.xlarge", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:39:01.910036 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:01.956098 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-a1cb1d6a-b47d-4e0d-92e8-f8b82f1f11ad admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.491205 Sep 30 09:39:01.956720 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 18/34] 10.222.0.39 () {64 vars in 1283 bytes} [Fri Sep 30 09:39:01 2022] POST /compute/v2.1/flavors => generated 408 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:39:01.969170 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-82cdbe59-80af-430b-8005-c7066661d497 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:39:02.053097 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-82cdbe59-80af-430b-8005-c7066661d497 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.087699 Sep 30 09:39:02.053919 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 17/35] 10.222.0.39 () {64 vars in 1333 bytes} [Fri Sep 30 09:39:01 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:39:18.881857 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-a1cb1d6a-b47d-4e0d-92e8-f8b82f1f11ad admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:18.882829 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-a1cb1d6a-b47d-4e0d-92e8-f8b82f1f11ad admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Sep 30 09:39:18.883229 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 19/36] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:39:18 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:18.893692 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:19.269053 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-57c51bd5-3a18-4134-9a63-96bb6672b5f8 admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:19.287088 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-57c51bd5-3a18-4134-9a63-96bb6672b5f8 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.397762 Sep 30 09:39:19.287631 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 18/37] 10.222.0.39 () {60 vars in 1292 bytes} [Fri Sep 30 09:39:18 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:39:20.582555 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-a1cb1d6a-b47d-4e0d-92e8-f8b82f1f11ad admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:20.583596 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-a1cb1d6a-b47d-4e0d-92e8-f8b82f1f11ad admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 Sep 30 09:39:20.583986 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 20/38] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:39:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:21.104282 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-958b317d-8982-4e6f-ba4d-69e2f6967bd6 admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:21.109568 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.wsgi [None req-958b317d-8982-4e6f-ba4d-69e2f6967bd6 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-958b317d-8982-4e6f-ba4d-69e2f6967bd6 admin admin] Request method failure captured: Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Accept: application/json Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Accept-Encoding: gzip, deflate Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Connection: keep-alive Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Content-Length: 0 Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Host: 10.222.0.39 Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Openstack-Api-Version: compute 2.92 Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Openstack-System-Scope: None Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: User-Agent: python-novaclient Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Auth-Token: gAAAAABjNrlIBtjdBRCEEMS5R4wP0eFATJ1MbasTWTDlJRF8LTOuGYa0KeG887OjhW9HHrTA6jyw-VHOXiB9VgZTfPwsb8eFBRl0teCU3iJM09Fwd5pA4d8pTvLwW5icCSUoRi9di8b22sYetN3u_hUftNgVJS0s74GfcVZV8kW0eVEtV0CPxwo Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Domain-Id: None Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Domain-Name: None Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Identity-Status: Confirmed Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Is-Admin-Project: True Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Openstack-Nova-Api-Version: 2.92 Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Domain-Id: default Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Domain-Name: Default Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Name: admin Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Role: admin,reader,member Sep 30 09:39:21.113430 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Roles: admin,reader,member Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant: admin Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant-Name: admin Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User: admin Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Domain-Id: default Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Domain-Name: Default Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Id: 767ee8fd8728440799a75a0b00b134c0 Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Name: admin Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: method: > Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: exception: Flavor 42 could not be found. Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: action_args: {'id': '42'} Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:39:21.115242 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:21.116438 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.117874 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-958b317d-8982-4e6f-ba4d-69e2f6967bd6 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=112417) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:39:21.119309 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-958b317d-8982-4e6f-ba4d-69e2f6967bd6 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.394239 Sep 30 09:39:21.119309 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 19/39] 10.222.0.39 () {60 vars in 1190 bytes} [Fri Sep 30 09:39:20 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 395 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:21.125001 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-095326d6-98b5-4ef5-9746-34d64c2bbdae admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:21.147082 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-095326d6-98b5-4ef5-9746-34d64c2bbdae admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.025906 Sep 30 09:39:21.148603 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 21/40] 10.222.0.39 () {60 vars in 1210 bytes} [Fri Sep 30 09:39:21 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:39:21.158634 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-3d132ae4-4200-44c0-8bef-b496be69256a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:21.180657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-3d132ae4-4200-44c0-8bef-b496be69256a admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026932 Sep 30 09:39:21.181206 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 20/41] 10.222.0.39 () {60 vars in 1210 bytes} [Fri Sep 30 09:39:21 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:39:21.190900 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-3a859e3b-f2b7-4109-b66a-822aad320954 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:21.196676 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.wsgi [None req-3a859e3b-f2b7-4109-b66a-822aad320954 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-3a859e3b-f2b7-4109-b66a-822aad320954 admin admin] Request method failure captured: Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Accept: application/json Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Accept-Encoding: gzip, deflate Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Connection: keep-alive Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Content-Length: 0 Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Host: 10.222.0.39 Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Openstack-Api-Version: compute 2.92 Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Openstack-System-Scope: None Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: User-Agent: python-novaclient Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Auth-Token: gAAAAABjNrlIBtjdBRCEEMS5R4wP0eFATJ1MbasTWTDlJRF8LTOuGYa0KeG887OjhW9HHrTA6jyw-VHOXiB9VgZTfPwsb8eFBRl0teCU3iJM09Fwd5pA4d8pTvLwW5icCSUoRi9di8b22sYetN3u_hUftNgVJS0s74GfcVZV8kW0eVEtV0CPxwo Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Domain-Id: None Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Domain-Name: None Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Identity-Status: Confirmed Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Is-Admin-Project: True Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Openstack-Nova-Api-Version: 2.92 Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Domain-Id: default Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Domain-Name: Default Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Name: admin Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Role: admin,reader,member Sep 30 09:39:21.199909 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Roles: admin,reader,member Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant: admin Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant-Name: admin Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User: admin Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Domain-Id: default Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Domain-Name: Default Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Id: 767ee8fd8728440799a75a0b00b134c0 Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Name: admin Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: method: > Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: exception: Flavor 42 could not be found. Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: action_args: {'id': '42'} Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:39:21.201548 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:21.206102 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:21.207564 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-3a859e3b-f2b7-4109-b66a-822aad320954 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=112418) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:39:21.208835 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-3a859e3b-f2b7-4109-b66a-822aad320954 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014297 Sep 30 09:39:21.208835 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 22/42] 10.222.0.39 () {60 vars in 1190 bytes} [Fri Sep 30 09:39:21 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:22.414619 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-3d132ae4-4200-44c0-8bef-b496be69256a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:22.415659 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-3d132ae4-4200-44c0-8bef-b496be69256a admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Sep 30 09:39:22.416057 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 21/43] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:39:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:22.932151 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-f219e11f-f302-4faf-9a47-604bd3f2a6d9 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:22.937142 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.wsgi [None req-f219e11f-f302-4faf-9a47-604bd3f2a6d9 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-f219e11f-f302-4faf-9a47-604bd3f2a6d9 admin admin] Request method failure captured: Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Accept: application/json Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Accept-Encoding: gzip, deflate Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Connection: keep-alive Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Content-Length: 0 Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Host: 10.222.0.39 Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Openstack-Api-Version: compute 2.92 Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Openstack-System-Scope: None Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: User-Agent: python-novaclient Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Auth-Token: gAAAAABjNrlK0PSTzrmpSLsGjhdPBVsSUgCvaZyMuyyTD_ZIvpRB6V_eCL67bVQowmS5BSj8gM0l6UHKnSEteJ3qn9Us4qOfNIlxeh74OdTXD2PEXqgVagsi_8HRm-meTD9Ix1T_xZWfEvaTEducJhi233k48UELD9XmcZcN8wdUE1PVyvGFR6o Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Domain-Id: None Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Domain-Name: None Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Identity-Status: Confirmed Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Is-Admin-Project: True Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Openstack-Nova-Api-Version: 2.92 Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Domain-Id: default Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Domain-Name: Default Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Name: admin Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Role: admin,reader,member Sep 30 09:39:22.938704 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Roles: admin,reader,member Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant: admin Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant-Name: admin Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User: admin Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Domain-Id: default Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Domain-Name: Default Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Id: 767ee8fd8728440799a75a0b00b134c0 Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Name: admin Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: method: > Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: exception: Flavor 84 could not be found. Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: action_args: {'id': '84'} Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:39:22.940237 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:22.941210 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:22.942755 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-f219e11f-f302-4faf-9a47-604bd3f2a6d9 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=112418) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:39:22.944000 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-f219e11f-f302-4faf-9a47-604bd3f2a6d9 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.387725 Sep 30 09:39:22.944000 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 23/44] 10.222.0.39 () {60 vars in 1190 bytes} [Fri Sep 30 09:39:22 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 389 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:22.949780 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-48a682ba-f8f4-462d-9c1b-75a348a6c019 admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:22.964948 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-48a682ba-f8f4-462d-9c1b-75a348a6c019 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018687 Sep 30 09:39:22.965470 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 22/45] 10.222.0.39 () {60 vars in 1210 bytes} [Fri Sep 30 09:39:22 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:39:22.975372 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-312067d2-1b58-4f2c-8c65-25715e057697 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:22.990490 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-312067d2-1b58-4f2c-8c65-25715e057697 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018611 Sep 30 09:39:22.991040 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 24/46] 10.222.0.39 () {60 vars in 1210 bytes} [Fri Sep 30 09:39:22 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:39:23.000021 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-3c8cca8e-b2ce-4d44-9e2a-8aad959d39de admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:23.004779 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.wsgi [None req-3c8cca8e-b2ce-4d44-9e2a-8aad959d39de admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-3c8cca8e-b2ce-4d44-9e2a-8aad959d39de admin admin] Request method failure captured: Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Accept: application/json Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Accept-Encoding: gzip, deflate Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Connection: keep-alive Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Content-Length: 0 Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Host: 10.222.0.39 Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Openstack-Api-Version: compute 2.92 Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Openstack-System-Scope: None Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: User-Agent: python-novaclient Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Auth-Token: gAAAAABjNrlK0PSTzrmpSLsGjhdPBVsSUgCvaZyMuyyTD_ZIvpRB6V_eCL67bVQowmS5BSj8gM0l6UHKnSEteJ3qn9Us4qOfNIlxeh74OdTXD2PEXqgVagsi_8HRm-meTD9Ix1T_xZWfEvaTEducJhi233k48UELD9XmcZcN8wdUE1PVyvGFR6o Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Domain-Id: None Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Domain-Name: None Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Identity-Status: Confirmed Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Is-Admin-Project: True Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Openstack-Nova-Api-Version: 2.92 Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Domain-Id: default Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Domain-Name: Default Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Name: admin Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Role: admin,reader,member Sep 30 09:39:23.006532 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Roles: admin,reader,member Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant: admin Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant-Name: admin Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User: admin Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Domain-Id: default Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Domain-Name: Default Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Id: 767ee8fd8728440799a75a0b00b134c0 Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Name: admin Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: method: > Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: exception: Flavor 84 could not be found. Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: action_args: {'id': '84'} Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:39:23.008122 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:23.009171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:23.010626 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-3c8cca8e-b2ce-4d44-9e2a-8aad959d39de admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=112417) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:39:23.011773 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-3c8cca8e-b2ce-4d44-9e2a-8aad959d39de admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011399 Sep 30 09:39:23.011773 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 23/47] 10.222.0.39 () {60 vars in 1190 bytes} [Fri Sep 30 09:39:22 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Sep 30 09:39:24.215883 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-312067d2-1b58-4f2c-8c65-25715e057697 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:24.216895 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-312067d2-1b58-4f2c-8c65-25715e057697 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001439 Sep 30 09:39:24.217287 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 25/48] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:39:24 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:24.727912 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-18c4b8d3-f498-4fed-a214-2f5fc7a0f4c4 admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:24.733645 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.wsgi [None req-18c4b8d3-f498-4fed-a214-2f5fc7a0f4c4 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-18c4b8d3-f498-4fed-a214-2f5fc7a0f4c4 admin admin] Request method failure captured: Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Accept: application/json Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Accept-Encoding: gzip, deflate Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Connection: keep-alive Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Content-Length: 0 Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Host: 10.222.0.39 Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Openstack-Api-Version: compute 2.92 Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Openstack-System-Scope: None Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: User-Agent: python-novaclient Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Auth-Token: gAAAAABjNrlMR824CG4hp6hxDeEKfVjMnkihInKXvq2S0nhGtPCg3TjELvzqjj7JWBIV3O2Q0F-QRSaeh5f7gzBUdZ2qy5FkxG-znenjrSVK_2FWFOriLhfJxdIv5I3J_gQb31J6OvJsrsBroqllfNsFw0jhxfa4zgf0QXpxuhvzU-Im0ruFVZo Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Domain-Id: None Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Domain-Name: None Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Identity-Status: Confirmed Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Is-Admin-Project: True Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Openstack-Nova-Api-Version: 2.92 Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Domain-Id: default Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Domain-Name: Default Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Project-Name: admin Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Role: admin,reader,member Sep 30 09:39:24.735171 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Roles: admin,reader,member Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant: admin Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-Tenant-Name: admin Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User: admin Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Domain-Id: default Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Domain-Name: Default Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Id: 767ee8fd8728440799a75a0b00b134c0 Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: X-User-Name: admin Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: method: > Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: exception: Flavor 451 could not be found. Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: action_args: {'id': '451'} Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:39:24.736738 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:24.737657 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.738744 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-18c4b8d3-f498-4fed-a214-2f5fc7a0f4c4 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=112417) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:39:24.739776 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-18c4b8d3-f498-4fed-a214-2f5fc7a0f4c4 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.382038 Sep 30 09:39:24.739776 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 24/49] 10.222.0.39 () {60 vars in 1193 bytes} [Fri Sep 30 09:39:24 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 383 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:24.746450 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-d41cdfe7-aef8-4ce0-9a23-597a5851f293 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:24.761865 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-d41cdfe7-aef8-4ce0-9a23-597a5851f293 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018988 Sep 30 09:39:24.762405 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 26/50] 10.222.0.39 () {60 vars in 1210 bytes} [Fri Sep 30 09:39:24 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:39:24.772406 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:24.787613 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018735 Sep 30 09:39:24.788146 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 25/51] 10.222.0.39 () {60 vars in 1210 bytes} [Fri Sep 30 09:39:24 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:39:24.797560 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-642fd00e-f5cb-4500-ae39-7610e3bf92dd admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:24.804794 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.wsgi [None req-642fd00e-f5cb-4500-ae39-7610e3bf92dd admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-642fd00e-f5cb-4500-ae39-7610e3bf92dd admin admin] Request method failure captured: Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Accept: application/json Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Accept-Encoding: gzip, deflate Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Connection: keep-alive Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Content-Length: 0 Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Host: 10.222.0.39 Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Openstack-Api-Version: compute 2.92 Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Openstack-System-Scope: None Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: User-Agent: python-novaclient Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Auth-Token: gAAAAABjNrlMR824CG4hp6hxDeEKfVjMnkihInKXvq2S0nhGtPCg3TjELvzqjj7JWBIV3O2Q0F-QRSaeh5f7gzBUdZ2qy5FkxG-znenjrSVK_2FWFOriLhfJxdIv5I3J_gQb31J6OvJsrsBroqllfNsFw0jhxfa4zgf0QXpxuhvzU-Im0ruFVZo Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Domain-Id: None Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Domain-Name: None Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Identity-Status: Confirmed Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Is-Admin-Project: True Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Openstack-Nova-Api-Version: 2.92 Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Domain-Id: default Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Domain-Name: Default Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Project-Name: admin Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Role: admin,reader,member Sep 30 09:39:24.806311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Roles: admin,reader,member Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f04f50d086c64df3bc18b4c8f8baadc4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant: admin Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant-Id: f04f50d086c64df3bc18b4c8f8baadc4 Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-Tenant-Name: admin Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User: admin Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Domain-Id: default Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Domain-Name: Default Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Id: 767ee8fd8728440799a75a0b00b134c0 Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: X-User-Name: admin Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: method: > Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: exception: Flavor 451 could not be found. Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: action_args: {'id': '451'} Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:39:24.807855 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:24.808840 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: ERROR nova.api.openstack.wsgi Sep 30 09:39:24.809942 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-642fd00e-f5cb-4500-ae39-7610e3bf92dd admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=112418) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:39:24.811020 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-642fd00e-f5cb-4500-ae39-7610e3bf92dd admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.013924 Sep 30 09:39:24.811020 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 27/52] 10.222.0.39 () {60 vars in 1193 bytes} [Fri Sep 30 09:39:24 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:26.008973 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:26.009992 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Sep 30 09:39:26.010380 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 26/53] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:39:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:26.530033 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-af850c6b-c906-4870-a508-21ed68f1022c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:39:26.623808 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-af850c6b-c906-4870-a508-21ed68f1022c admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.474824 Sep 30 09:39:26.624366 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 28/54] 10.222.0.39 () {64 vars in 1235 bytes} [Fri Sep 30 09:39:26 2022] POST /compute/v2.1/flavors => generated 404 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:39:27.841061 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:27.842071 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001579 Sep 30 09:39:27.842468 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 27/55] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:39:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:28.358664 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-5e4d39af-b032-4ec2-8f8c-1f4cbe5a757c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:39:28.429159 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-5e4d39af-b032-4ec2-8f8c-1f4cbe5a757c admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.450186 Sep 30 09:39:28.429713 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 29/56] 10.222.0.39 () {64 vars in 1235 bytes} [Fri Sep 30 09:39:27 2022] POST /compute/v2.1/flavors => generated 405 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:39:29.634770 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:29.635790 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001484 Sep 30 09:39:29.636208 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 28/57] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:39:29 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:30.152057 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-b6ccdeb1-ba29-4954-85d9-cb277e4e5b11 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:39:30.224655 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-b6ccdeb1-ba29-4954-85d9-cb277e4e5b11 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.452799 Sep 30 09:39:30.225193 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 30/58] 10.222.0.39 () {64 vars in 1235 bytes} [Fri Sep 30 09:39:29 2022] POST /compute/v2.1/flavors => generated 407 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:39:31.432078 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:31.433096 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Sep 30 09:39:31.433498 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 29/59] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:39:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:31.948483 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-ce5a644c-9b15-4ada-8996-766aacb8d9ba admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:39:32.006023 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-ce5a644c-9b15-4ada-8996-766aacb8d9ba admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.436189 Sep 30 09:39:32.006569 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 31/60] 10.222.0.39 () {64 vars in 1235 bytes} [Fri Sep 30 09:39:31 2022] POST /compute/v2.1/flavors => generated 409 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:39:33.207056 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:33.208069 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001469 Sep 30 09:39:33.208463 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 30/61] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:39:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:39:33.720385 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-c27e4268-f307-4cc3-9b0f-b39811be7620 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:39:33.740307 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-c27e4268-f307-4cc3-9b0f-b39811be7620 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.395697 Sep 30 09:39:33.740839 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 32/62] 10.222.0.39 () {60 vars in 1202 bytes} [Fri Sep 30 09:39:33 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:40:29.906828 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:40:29.907855 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-10377e3f-7016-401f-9650-9ada58f0858a admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001507 Sep 30 09:40:29.908260 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 31/63] 10.222.0.39 () {54 vars in 931 bytes} [Fri Sep 30 09:40:29 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:40:29.918782 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:40:30.296339 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-4048231b-12f6-4e06-a5b5-b11c195edfdc admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:40:30.316235 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-4048231b-12f6-4e06-a5b5-b11c195edfdc admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.401918 Sep 30 09:40:30.316804 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 33/64] 10.222.0.39 () {60 vars in 1292 bytes} [Fri Sep 30 09:40:29 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:40:30.337203 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writing DB stats nova_api:SELECT=14 {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:40:30.340906 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-24198649-8afb-48d6-92c2-8b44ef2b6b6c admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:40:30.348429 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-24198649-8afb-48d6-92c2-8b44ef2b6b6c admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012657 Sep 30 09:40:30.349039 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 32/65] 10.222.0.39 () {60 vars in 1283 bytes} [Fri Sep 30 09:40:30 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:40:30.360977 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-2e3d2df0-457e-4c8f-9df3-fbea7882b0ab admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:40:30.368083 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-2e3d2df0-457e-4c8f-9df3-fbea7882b0ab admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011247 Sep 30 09:40:30.368612 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 34/66] 10.222.0.39 () {60 vars in 1286 bytes} [Fri Sep 30 09:40:30 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:40:30.380114 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:40:30.387275 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011165 Sep 30 09:40:30.387781 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 33/67] 10.222.0.39 () {60 vars in 1286 bytes} [Fri Sep 30 09:40:30 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:40:30.399121 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-4a5606aa-45d5-4d27-a564-6f36d61026e4 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:40:30.406805 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-4a5606aa-45d5-4d27-a564-6f36d61026e4 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011096 Sep 30 09:40:30.407311 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 35/68] 10.222.0.39 () {60 vars in 1283 bytes} [Fri Sep 30 09:40:30 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:43:54.485734 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:43:54.486816 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Sep 30 09:43:54.487211 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 34/69] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:43:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:43:54.629524 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_api:SELECT=6 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:54.769802 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-69ff119f-f92d-4517-97d4-7a38ca8fab14 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:43:54.774484 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-69ff119f-f92d-4517-97d4-7a38ca8fab14 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:43:54.775003 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-69ff119f-f92d-4517-97d4-7a38ca8fab14 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:43:54.775433 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-69ff119f-f92d-4517-97d4-7a38ca8fab14 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:43:54.776139 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-69ff119f-f92d-4517-97d4-7a38ca8fab14 admin admin] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:43:54.776577 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-69ff119f-f92d-4517-97d4-7a38ca8fab14 admin admin] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:43:54.776985 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-69ff119f-f92d-4517-97d4-7a38ca8fab14 admin admin] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:43:54.807487 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-69ff119f-f92d-4517-97d4-7a38ca8fab14 admin admin] 10.222.0.39 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.179107 Sep 30 09:43:54.808025 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 36/70] 10.222.0.39 () {60 vars in 1193 bytes} [Fri Sep 30 09:43:54 2022] GET /compute/v2.1/os-services => generated 791 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:43:55.877870 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:43:55.878974 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Sep 30 09:43:55.879372 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 35/71] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:43:55 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:43:56.069443 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-6ab92cfd-e286-41ea-8b3e-339cfefd2609 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:43:56.073026 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-6ab92cfd-e286-41ea-8b3e-339cfefd2609 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:43:56.073498 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-6ab92cfd-e286-41ea-8b3e-339cfefd2609 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:43:56.073923 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-6ab92cfd-e286-41ea-8b3e-339cfefd2609 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:43:56.074656 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-6ab92cfd-e286-41ea-8b3e-339cfefd2609 admin admin] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:43:56.075132 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-6ab92cfd-e286-41ea-8b3e-339cfefd2609 admin admin] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:43:56.075544 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-6ab92cfd-e286-41ea-8b3e-339cfefd2609 admin admin] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:43:56.094970 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-6ab92cfd-e286-41ea-8b3e-339cfefd2609 admin admin] 10.222.0.39 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.078128 Sep 30 09:43:56.095447 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 37/72] 10.222.0.39 () {60 vars in 1193 bytes} [Fri Sep 30 09:43:56 2022] GET /compute/v2.1/os-services => generated 791 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:44:07.578843 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:44:07.579864 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001479 Sep 30 09:44:07.580287 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 36/73] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:44:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:44:07.720309 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_cell0:SELECT=4 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:07.721558 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_cell1:SELECT=4 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:07.722224 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG dbcounter [-] [112418] Writing DB stats nova_api:SELECT=8 {{(pid=112418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:07.770225 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-73f41c74-aa68-45b0-8a7b-3ea18c454080 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:44:07.778938 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-73f41c74-aa68-45b0-8a7b-3ea18c454080 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:44:07.778938 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-73f41c74-aa68-45b0-8a7b-3ea18c454080 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:44:07.778938 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-73f41c74-aa68-45b0-8a7b-3ea18c454080 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:44:07.778938 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-73f41c74-aa68-45b0-8a7b-3ea18c454080 admin admin] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:44:07.778938 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-73f41c74-aa68-45b0-8a7b-3ea18c454080 admin admin] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:44:07.779421 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-73f41c74-aa68-45b0-8a7b-3ea18c454080 admin admin] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:44:07.802354 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-73f41c74-aa68-45b0-8a7b-3ea18c454080 admin admin] 10.222.0.39 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.086269 Sep 30 09:44:07.802869 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 38/74] 10.222.0.39 () {60 vars in 1193 bytes} [Fri Sep 30 09:44:07 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:44:08.873828 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG nova.api.openstack.wsgi [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] Calling method '>' {{(pid=112417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:44:08.875166 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: INFO nova.api.openstack.requestlog [None req-cb0f6bb2-c488-4a10-a094-d8b19a85ceaa admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001528 Sep 30 09:44:08.875561 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: [pid: 112417|app: 0|req: 37/75] 10.222.0.39 () {56 vars in 1074 bytes} [Fri Sep 30 09:44:08 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:44:09.062257 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG nova.api.openstack.wsgi [None req-a032594b-b6e0-41ad-b10d-393596887b53 admin admin] Calling method '>' {{(pid=112418) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:44:09.066114 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-a032594b-b6e0-41ad-b10d-393596887b53 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:44:09.066608 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-a032594b-b6e0-41ad-b10d-393596887b53 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:44:09.067033 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-a032594b-b6e0-41ad-b10d-393596887b53 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:44:09.067742 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-a032594b-b6e0-41ad-b10d-393596887b53 admin admin] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:44:09.068183 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-a032594b-b6e0-41ad-b10d-393596887b53 admin admin] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:44:09.068597 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: DEBUG oslo_concurrency.lockutils [None req-a032594b-b6e0-41ad-b10d-393596887b53 admin admin] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112418) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:44:09.090443 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: INFO nova.api.openstack.requestlog [None req-a032594b-b6e0-41ad-b10d-393596887b53 admin admin] 10.222.0.39 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.079599 Sep 30 09:44:09.091013 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: [pid: 112418|app: 0|req: 39/76] 10.222.0.39 () {60 vars in 1193 bytes} [Fri Sep 30 09:44:09 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:45:04.276536 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: Fri Sep 30 09:45:04 2022 - graceful shutdown triggered... Sep 30 09:45:04.276645 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 30 09:45:04.277610 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: Gracefully killing worker 1 (pid: 112417)... Sep 30 09:45:04.278029 dv-cinder-master-3-iscsi devstack@n-api.service[112418]: Gracefully killing worker 2 (pid: 112418)... Sep 30 09:45:04.403759 dv-cinder-master-3-iscsi devstack@n-api.service[112417]: DEBUG dbcounter [-] [112417] Writing DB stats nova_api:SELECT=4 {{(pid=112417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:05.277936 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: worker 1 buried after 1 seconds Sep 30 09:45:05.278473 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: worker 2 buried after 1 seconds Sep 30 09:45:05.278473 dv-cinder-master-3-iscsi devstack@n-api.service[112416]: goodbye to uWSGI. Sep 30 09:45:05.279608 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 30 09:45:05.280158 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 30 09:45:05.283006 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 30 09:45:05.292251 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 30 09:45:05 2022] *** Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: nodename: dv-cinder-master-3-iscsi Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: machine: x86_64 Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: clock source: unix Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: pcre jit disabled Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: detected number of CPU cores: 4 Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: current working directory: / Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: detected binary path: /usr/bin/uwsgi-core Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: your processes number limit is 47680 Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: your memory page size is 4096 bytes Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: detected max file descriptor number: 2048 Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: lock engine: pthread robust mutexes Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: thunder lock: enabled Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 30 09:45:05.296083 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 30 09:45:05.335693 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: Python main interpreter initialized at 0x55badf9ea9c0 Sep 30 09:45:05.335693 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: python threads support enabled Sep 30 09:45:05.335693 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: your server socket listen backlog is limited to 100 connections Sep 30 09:45:05.335693 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: your mercy for graceful operations on workers is 90 seconds Sep 30 09:45:05.336095 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: mapped 403077 bytes (393 KB) for 2 cores Sep 30 09:45:05.336179 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: *** Operational MODE: preforking *** Sep 30 09:45:05.336418 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: *** uWSGI is running in multiple interpreter mode *** Sep 30 09:45:05.336418 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: spawned uWSGI master process (pid: 127438) Sep 30 09:45:05.336621 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 30 09:45:05.339331 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: spawned uWSGI worker 1 (pid: 127455, cores: 1) Sep 30 09:45:05.339331 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: spawned uWSGI worker 2 (pid: 127456, cores: 1) Sep 30 09:45:05.339331 dv-cinder-master-3-iscsi devstack@n-api.service[127438]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 30 09:45:05.997646 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 30 09:45:05.998537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 30 09:45:07.159647 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:45:07.159647 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: from cryptography.utils import int_from_bytes Sep 30 09:45:07.160844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:45:07.160844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: from cryptography.utils import int_from_bytes Sep 30 09:45:07.165212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:45:07.165212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: from cryptography.utils import int_from_bytes Sep 30 09:45:07.165698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:45:07.165698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: from cryptography.utils import int_from_bytes Sep 30 09:45:09.553133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 30 09:45:09.553731 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 30 09:45:09.554052 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 30 09:45:09.554287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 30 09:45:09.554540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 30 09:45:09.554951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.555279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.555601 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.555922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.556256 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.556562 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.556881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.557197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.557522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.557777 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.558090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.558561 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.558910 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.559221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.559534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.559852 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.559959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 30 09:45:09.560327 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.560403 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 30 09:45:09.560485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.560731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 30 09:45:09.560844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.561094 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 30 09:45:09.561261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.561379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 30 09:45:09.568007 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568167 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568863 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.568944 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569110 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569418 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569502 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569575 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569655 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569737 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569912 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.569984 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.570060 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.570132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.570208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.570293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.570372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.570372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.570513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.570585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.571180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.571341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.571494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.571647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.571823 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.571823 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.572127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.572354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.572527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.572675 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.572829 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.572981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.573143 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.573288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.573432 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.573597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.573742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.573896 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.574055 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.574209 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.574363 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.574518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.574697 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.574846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.575003 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.575157 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.575310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.575462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.575462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.575768 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.575981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.576145 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.576291 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.576487 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.576652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.576806 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.576958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.577113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.577273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.578708 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.578901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.579060 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.579214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.580021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.580189 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.580352 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.580500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.580654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.580810 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.580961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.581116 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.581116 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.581426 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.581580 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.581724 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.581877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.582029 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.582183 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.582336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.582481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.582697 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.582849 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.582994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.583151 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.583304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.583467 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.583625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.583625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.583917 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.584062 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.584213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.584373 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.584524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.39 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.584669 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.584822 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.39 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.584967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.585118 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.585273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.585428 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.585574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.585734 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.585887 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.586044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.586197 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.586342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.586537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.586713 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.586860 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.587003 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.587167 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.587312 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.587466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.588309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.588504 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.588842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.589005 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.589161 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.589316 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.589461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.589605 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.589749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.589917 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.590145 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.590303 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.590457 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.590637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.590800 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.590947 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.591100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.591253 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.591405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.591557 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.591701 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.591844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.39 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.592014 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.592168 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.39 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.592323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.592467 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.592617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.592762 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.592924 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.593069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.593220 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.593366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.593517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.593670 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.593814 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.593814 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.594108 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.594258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.594410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.594567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.594747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.594893 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.595060 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.595217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.595369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.595532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.595684 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.595837 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.595998 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.595998 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.596280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.596438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.596624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.596782 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.596940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.597085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.597238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.597391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.597543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.597708 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.597863 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.598485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.598949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.599117 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.599277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.599439 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.599591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.599758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.599912 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.600202 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.600358 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.600513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.600665 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.600817 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.600983 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.601136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.601288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.601444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.601589 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.601731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.601881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.602040 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.602185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.602341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.602498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.602660 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.602829 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.602975 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.603133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.603279 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.603423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.603583 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.603736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.603890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.604035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.604178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.604337 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.604494 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.604647 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.604801 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.604954 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.605106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.605257 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.605413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.605566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.605711 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.605853 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.606004 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.606148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.606304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.606457 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.606641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.606854 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.607008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.607170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.607332 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.607485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.607642 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.607792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.607936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.608089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.608690 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.608844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.609183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.609346 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.609491 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.609645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.609792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.609936 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.610080 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.610251 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.610463 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.611135 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.611299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.611444 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.611588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.611740 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.611907 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.612053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.612198 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.612349 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.612501 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.612647 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.612804 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.612958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.613103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.613103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.613385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.613537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.613696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.613851 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:45:09.613997 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.614141 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.614293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.614445 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.614624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.614624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.615041 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.615186 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.615340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.615499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.615646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.615646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.615951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.616097 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.616255 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.616408 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.616562 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.616715 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.616903 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.617051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.617208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.617353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.617577 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.617818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.618001 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.618157 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.618401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.618983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.619347 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.619511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.619657 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.619807 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.619953 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.620095 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.620253 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.620399 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.620630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.620798 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.620945 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.621096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.621255 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.621408 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.621561 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.621714 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.621867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.622026 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.622171 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.622322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.622476 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.622650 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.622801 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.622959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.623353 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.623508 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.623660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.623805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.623956 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.624114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.624267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.624420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.624567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.624719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.624874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.625033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.625178 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.625334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.625480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.625480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.625773 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.625926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.626101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.626936 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.627099 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.627244 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.627402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.627561 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.627726 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.627883 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.628028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.628180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.628334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.628334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.628735 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.629240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.629589 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.629751 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.629907 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.629907 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.630201 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.630346 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.630499 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.630681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.630905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.631054 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.631197 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.631340 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.631492 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.631652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.631799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.631951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.632105 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.632260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.632412 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.632570 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.632722 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.632876 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.633028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.633173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.633398 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.633576 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.633725 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.633878 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.634030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.634185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.634339 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.634483 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.634667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.634828 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.634975 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.635119 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.635273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.635419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.635571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.635724 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.635877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.636022 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.636191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.636191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.636473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.636618 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.636769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.636913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.637114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.637273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.637418 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.637561 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.637712 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.637874 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.638020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.638179 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.638324 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.638467 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.638722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.638883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.639376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.639376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.639859 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.640007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.640152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.640313 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.640460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.640611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.640775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.640945 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.641166 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.641328 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.641483 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.641629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.641773 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.641958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.642116 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.642270 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.642415 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.642559 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.642738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.642885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.643045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.643191 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.643344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.643496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.643647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.643800 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.643956 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.644108 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.644260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.644405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.644549 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.644711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.644866 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.645025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.645178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.645323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.645477 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.645636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.645781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.645933 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.646085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.646085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.646360 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.646514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.646722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.646879 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.647023 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.647208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.647357 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.647511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.647665 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.647818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.647963 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.648115 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.648259 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.648413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.648560 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.648741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.648957 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.649125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.649608 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.650045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.650212 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.650360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.650504 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.650667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.650833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.650980 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.651150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.651377 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.651535 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.651681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.651837 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.651991 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.652143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.652288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.652439 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.652591 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.652751 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.652898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.653051 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.653197 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.653352 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.653496 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.653654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.653808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.653964 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.654109 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.654337 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.654511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.654720 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.654880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.655143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.655314 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.655462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.655635 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.655782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.655937 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.656091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.656236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.656388 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.656546 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.656691 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.656843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.656989 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.657133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.657331 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.657527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.657688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.657843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.657989 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.658134 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.658287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.658433 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.658602 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.658766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.658912 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.659152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.659312 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.659832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.660188 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.660352 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.660508 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.660660 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.660823 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.660977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.661121 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.661266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.661505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.661656 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.661817 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.661817 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.662099 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.662255 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.662399 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.662542 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.662764 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.662920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.663066 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.663217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.663362 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.663515 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.663668 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.663821 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.663965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.664108 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.664263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.664428 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.664574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.664718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.664862 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.665013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.665165 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.665320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.665478 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.665623 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.665766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.665918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666070 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666236 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666536 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666718 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.666872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.667620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.667776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.667931 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.668076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.668230 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.668381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.668535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.668681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.668838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.668984 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.669153 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.669376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.669376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.674056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.674241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.674409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.674573 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.674768 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.674935 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.675098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.675259 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.675419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.675578 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.675729 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.675889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.676056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.676223 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.676383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.676542 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.676714 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.676874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.677040 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.677200 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.677351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.677568 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.677725 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.677884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.678046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.678214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.678369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.678369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.678747 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.678931 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.679105 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.679366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.679603 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.679761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.679922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.680125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.680287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.680450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.680637 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.681220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.681409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.681646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.681817 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.681977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.682141 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.682304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.682467 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.682651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.682818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.682971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.683131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.683284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.683435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.683594 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.683746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.683933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.684170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.684330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.684481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.684632 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.684782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.684933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.685093 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.685246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.685396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.685546 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.685697 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.685871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.686034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.686194 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.686355 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.686506 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.686678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.686917 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.687073 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.687224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.687376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.687376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.687731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.687904 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.688067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.688225 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.688387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.688548 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.688723 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.689407 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.689407 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.689799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.689965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.690131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.690292 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.690444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.690651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.690832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.691422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.691594 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.691594 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.691999 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.692161 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.692323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.692476 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.692640 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.692793 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.692952 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.693103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.693279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.693432 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.693583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.693744 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.693897 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.694057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.694378 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.694547 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.694734 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.694896 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.695060 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.695220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.695387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.695551 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.695702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.695860 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.696011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.696176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.696342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.696503 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.696655 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.696807 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.696966 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.697130 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.697290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.698020 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.698204 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.698370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.698524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.698730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.698895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.699054 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.699206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.699365 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.699517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.699677 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.699916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.699916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.700220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.700390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.700550 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.700712 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.700865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.701027 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.701523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.701523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.701523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.702177 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.702351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.702512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.702716 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.702883 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.703050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.703212 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.703370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.703529 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.703680 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.703680 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.703990 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.704143 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.704320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.704574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.704731 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.704897 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.705068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.705222 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.705372 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.705530 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.705682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.705842 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.705994 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.706152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.706305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.706463 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.706642 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.706813 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.706974 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.707136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.707136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.707433 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.707585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.707746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.707898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.708090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.708260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.708417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.708581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.708742 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.708901 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.709061 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.709223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.709383 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.709557 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.709709 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.709899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.710131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.710290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.710442 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.710645 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.710806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.711021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.711179 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.711698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.711870 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.712034 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.712379 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.712539 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.712692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.712844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.713010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.713164 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.713315 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.713474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.713628 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.713781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.713961 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.714107 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.714220 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.714331 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.714451 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.714601 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.726422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.726656 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.726812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.726956 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.727088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.727229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.727371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.727511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.727511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.727774 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.727913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.728044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.728239 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.728374 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.728502 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.728638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.728776 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.728920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.729049 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.729185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.729315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.729444 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.729571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.729709 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.729846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.729975 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.730103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.730246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.730452 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.730612 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.730758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.730892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.731069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.731210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.731342 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.731489 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.731621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.731770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.732235 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.732368 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.732701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.733410 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.733552 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.733695 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.733828 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.733958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.734096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.734228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.734366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.734754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.734925 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.735151 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.735295 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.735471 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.736133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.736271 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.736410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.736550 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.736692 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.736824 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.736970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.737111 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.737244 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.737381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.737522 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.737655 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.737875 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.738010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.738141 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.738329 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.738473 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.738629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.738780 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.738904 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.739016 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.739127 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.739239 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.739380 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.739512 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.739512 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.739769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.739909 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.740048 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.740194 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.740334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.740487 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.740700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.740835 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.740965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.741116 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.741257 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.741388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.741527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.741666 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.741797 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.741958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.742504 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.742969 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.742969 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.743247 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.743393 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.743527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.743656 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.743793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.743933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744065 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744204 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744343 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744603 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.744733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.745811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.745811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.745811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.745811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.745811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.745811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.746827 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.746959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747313 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747408 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747814 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747888 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.747970 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.748054 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.748463 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.748567 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.748694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.748823 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.748823 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.748995 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749356 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749451 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749813 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.749913 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750010 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750104 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750189 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750365 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750459 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750553 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750675 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750777 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750864 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.750959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.751050 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.751133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.751266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.751266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.751932 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.752028 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.752133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753104 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753104 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753386 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753481 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753743 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.753931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.754024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.754109 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.754201 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.754287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.754370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.754370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.754526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.754628 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761063 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761272 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761475 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761745 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.761884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.762062 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.762157 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.762252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.762252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.762252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.762866 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.762963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763508 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763595 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763595 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763841 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.763841 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764006 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764091 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764175 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764429 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764686 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764770 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764853 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.764951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765038 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765122 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765122 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765487 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765590 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765676 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765759 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765922 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.765922 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766098 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766343 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766813 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766937 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.766937 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767110 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767202 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767295 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767472 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.767957 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.768933 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769020 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769104 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769368 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769785 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769785 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.769952 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770216 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770515 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770715 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770803 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.770888 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.772763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.772763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.772959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773123 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773559 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773559 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773690 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773762 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.773930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774005 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774291 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774455 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774798 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774880 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.774968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.775059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.775154 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.776445 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.776543 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.776641 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.776736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.776812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.776884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.776965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777052 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777127 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777603 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.777682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778006 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778125 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778346 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778879 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.778968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.779053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.779053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.779216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.779304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.779879 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.779978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780066 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780066 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780243 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780331 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780424 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780517 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780603 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780694 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780694 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780898 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.780991 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781167 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781254 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781352 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781438 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781522 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781697 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781794 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.781965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782048 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782362 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782478 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782851 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.782941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783042 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783213 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783825 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.783929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784026 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784211 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784585 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784585 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.784923 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785008 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785200 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785382 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785462 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785535 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785714 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785807 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.785981 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786080 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786166 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786643 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786723 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786797 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.786968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787235 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787319 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787504 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787599 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.787958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788138 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788225 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788400 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788400 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788754 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.788930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789095 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789195 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789476 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789568 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789661 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.789987 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.790080 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.790185 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.790272 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.790356 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.790356 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.790509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.790615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.795620 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.795731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.795819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.795912 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.795912 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796061 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796061 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796315 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796405 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796405 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796405 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796405 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.796923 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797013 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797116 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797294 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797477 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797666 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797751 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.797937 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.798029 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.798787 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.798895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.798982 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799080 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799265 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799356 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799634 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799726 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.799980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800072 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800599 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800690 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800783 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800868 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800868 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.800868 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801108 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801284 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801377 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801553 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801646 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801822 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801822 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.801990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802073 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802156 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802248 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802439 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802525 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802824 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.802911 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803060 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803302 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803598 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803740 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.803955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804239 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804326 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804662 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804760 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804760 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.804917 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805000 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805169 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805329 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805425 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805510 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805770 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.805977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806117 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806211 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806296 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806296 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806709 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.806839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807016 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807468 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807750 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807837 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.807921 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808149 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808386 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808497 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808608 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808795 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.808986 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809070 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809154 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809499 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809787 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809882 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.809977 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810074 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810167 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810633 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810720 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.810934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811172 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811395 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811868 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.811988 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812111 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812466 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812785 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812871 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.812971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.813065 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.813192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.813295 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.813389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.813485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.813578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.813671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.814340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.814459 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.814562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.814692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.814790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.814877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.814985 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.814985 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.815153 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.815249 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.815336 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.815432 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.815525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.815611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.815690 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816285 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816469 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816735 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.816937 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817140 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817237 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817332 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817424 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817597 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817836 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817908 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.817987 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.818074 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.818155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.818229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.818314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.818399 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.818483 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.818581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.818688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.822733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.822901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.822997 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823092 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823092 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823311 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823601 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823686 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823780 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823867 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.823951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.824179 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.824286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.824374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.824502 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.824623 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.824740 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.824842 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.824940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825214 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825306 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825502 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825673 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825757 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.825890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826062 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826486 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826701 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826887 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.826976 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827061 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827146 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827770 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827864 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.827951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828472 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828472 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828793 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.828887 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829695 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829787 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.829967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830054 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830146 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830239 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830510 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830750 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830847 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.830934 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127456) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 30 09:45:09.831020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.831800 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.832095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.832413 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.832759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.833091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.833409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.833727 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.834066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:45:09.834311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 30 09:45:09.892564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO dbcounter [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Registered counter for database nova_api Sep 30 09:45:09.897250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO dbcounter [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Registered counter for database nova_api Sep 30 09:45:09.913072 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_db.sqlalchemy.engines [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127456) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:45:09.917683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_db.sqlalchemy.engines [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127455) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:45:09.920077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writer thread running {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:45:09.924940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writer thread running {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:45:09.977200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.context [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),f6355185-7b23-453c-870b-129ec40cec19(cell1) {{(pid=127456) load_cells /opt/stack/nova/nova/context.py:464}} Sep 30 09:45:09.981193 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:45:09.981433 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.context [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),f6355185-7b23-453c-870b-129ec40cec19(cell1) {{(pid=127455) load_cells /opt/stack/nova/nova/context.py:464}} Sep 30 09:45:09.981665 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:45:09.983281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:45:09.983997 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:45:09.984439 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:45:09.985419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:45:09.986455 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:45:09.987124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:45:09.988082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:45:09.988813 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:45:09.989075 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO dbcounter [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Registered counter for database nova_cell0 Sep 30 09:45:09.989260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:45:09.991308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:45:09.993133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO dbcounter [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Registered counter for database nova_cell1 Sep 30 09:45:09.993271 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO dbcounter [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Registered counter for database nova_cell0 Sep 30 09:45:09.997283 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO dbcounter [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Registered counter for database nova_cell1 Sep 30 09:45:10.000993 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_db.sqlalchemy.engines [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127456) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:45:10.003024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_db.sqlalchemy.engines [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127456) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:45:10.004908 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_db.sqlalchemy.engines [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127455) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:45:10.007095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_db.sqlalchemy.engines [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127455) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:45:10.011729 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writer thread running {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:45:10.012111 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writer thread running {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:45:10.013742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writer thread running {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:45:10.014739 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writer thread running {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:45:10.163035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO dbcounter [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Registered counter for database nova_cell0 Sep 30 09:45:10.163841 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO dbcounter [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Registered counter for database nova_cell0 Sep 30 09:45:10.170383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_db.sqlalchemy.engines [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127456) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:45:10.174716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_db.sqlalchemy.engines [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127455) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:45:10.175832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writer thread running {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:45:10.182454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writer thread running {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:45:10.496046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: WARNING keystonemiddleware.auth_token [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 30 09:45:10.503892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: WARNING keystonemiddleware.auth_token [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 30 09:45:10.507012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: WARNING keystonemiddleware.auth_token [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 30 09:45:10.514260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: WARNING keystonemiddleware.auth_token [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 30 09:45:10.853235 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: WARNING keystonemiddleware.auth_token [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 30 09:45:10.859835 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: WARNING keystonemiddleware.auth_token [None req-4a9c09e2-89a3-475f-8bdb-e15572bf9747 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 30 09:45:10.860034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: WARNING keystonemiddleware.auth_token [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 30 09:45:10.865240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55badf9ea9c0 pid: 127456 (default app) Sep 30 09:45:10.866057 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: WARNING keystonemiddleware.auth_token [None req-51950c15-387c-4be2-bdd4-2f167d540433 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 30 09:45:10.871444 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55badf9ea9c0 pid: 127455 (default app) Sep 30 09:45:51.007458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-882221f5-8ba4-4e7a-a493-c5657f72c36d tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c1bb80d0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:45:51.038848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:51.039641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:51.040454 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:51.041162 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:52.375458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-882221f5-8ba4-4e7a-a493-c5657f72c36d tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d72489fc-f37f-4a31-a35e-8a3da69bd9ff" status: 200 len: 350 microversion: 2.1 time: 1.743546 Sep 30 09:45:52.376922 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 1/1] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:45:50 2022] GET /compute/v2.1/os-volumes/d72489fc-f37f-4a31-a35e-8a3da69bd9ff => generated 350 bytes in 1747 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:45:55.367366 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:55.369057 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:55.371773 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:55.375444 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:55.723472 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e30d26f5-d0e6-4188-b871-176d09fba954 tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c1b7b1f0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:45:56.288602 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e30d26f5-d0e6-4188-b871-176d09fba954 tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/9d8d15b1-abe1-4219-bfab-2e6cb317c503" status: 200 len: 351 microversion: 2.1 time: 0.927298 Sep 30 09:45:56.290637 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 1/2] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:45:55 2022] GET /compute/v2.1/os-volumes/9d8d15b1-abe1-4219-bfab-2e6cb317c503 => generated 351 bytes in 932 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:45:56.520887 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1679196727", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:45:56.804478 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] validate_networks() for [('069f8527-f158-4f06-809a-7a507bf3fc84', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:45:58.516740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:45:58.516740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:45:58.516740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:45:58.516740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:45:58.516740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:45:58.543240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:45:58.543930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:45:58.544480 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:45:58.545087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:45:58.545554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:45:58.545984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:45:58.628674 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Getting quotas for project f1c0948ddb014e6f93d6069bb5203890. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:45:58.642819 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Getting quotas for user 7ad21da05fc246d489b0afa641f20216 and project f1c0948ddb014e6f93d6069bb5203890. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:45:58.659507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:45:58.670251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:45:58.972360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c735f514-b4e9-423c-8f6c-14e11a2f5309 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.629679 Sep 30 09:45:58.975589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 2/3] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:45:56 2022] POST /compute/v2.1/servers => generated 383 bytes in 2634 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:45:58.988990 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-58a129e2-1058-4875-bcbe-9ea6870a1571 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:45:58.989884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-58a129e2-1058-4875-bcbe-9ea6870a1571 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:45:59.349153 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-58a129e2-1058-4875-bcbe-9ea6870a1571 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:45:59.390029 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-58a129e2-1058-4875-bcbe-9ea6870a1571 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1222 microversion: 2.1 time: 0.405739 Sep 30 09:45:59.393032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:45:58 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1222 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:45:59.736648 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1127181935", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:45:59.857954 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] validate_networks() for [('069f8527-f158-4f06-809a-7a507bf3fc84', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:46:00.679038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:46:00.680257 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:46:00.681394 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:46:00.682030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:46:00.683566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:46:00.726853 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:00.728291 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:00.743652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:00.743652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:00.743652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:00.743652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:00.842334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Getting quotas for project f1c0948ddb014e6f93d6069bb5203890. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:46:00.882103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Getting quotas for user 7ad21da05fc246d489b0afa641f20216 and project f1c0948ddb014e6f93d6069bb5203890. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:46:00.882507 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-779446128", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "networks": [{"uuid": "5ccfff4c-afb9-44a9-8e8c-e57c26fbfae3"}]}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:00.921111 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:46:00.935077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:46:01.167396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e4e499bc-8315-4bfc-a011-88f54db1a1a3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.437403 Sep 30 09:46:01.180870 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 3/5] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:45:59 2022] POST /compute/v2.1/servers => generated 383 bytes in 1452 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:46:01.195072 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] validate_networks() for [('5ccfff4c-afb9-44a9-8e8c-e57c26fbfae3', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:46:01.201436 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bcaa6b57-49a6-480d-a976-a3012e513c3b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:01.202089 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bcaa6b57-49a6-480d-a976-a3012e513c3b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:01.639032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bcaa6b57-49a6-480d-a976-a3012e513c3b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:01.676798 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bcaa6b57-49a6-480d-a976-a3012e513c3b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1222 microversion: 2.1 time: 0.488439 Sep 30 09:46:01.695179 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 4/6] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:01 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1222 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:02.183559 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f5e0af25-d731-4633-99a6-fe2857ac21da tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: .version_select at 0x7f72c10c31f0>, body: {"keypair": {"name": "tempest-keypair-2119459492", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK5tJ6XXtmG11dKSsWSF6WSC4niiS+e7r6v+PGrNPBZlhdzRNnOb1qSEMJ98lpdNEdTF5qL+v9M2UJcPbIt73ldVODmTKhQmPBTgsh3rBx597ZympUzCvM+MNjOTpBMmGg=="}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:02.235868 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-f5e0af25-d731-4633-99a6-fe2857ac21da tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for project 86e6b0501cb94028be07bddd2182638b. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:46:02.244604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-f5e0af25-d731-4633-99a6-fe2857ac21da tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for user 8c4ced79c3ff4842b30b472acb555cd5 and project 86e6b0501cb94028be07bddd2182638b. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:46:02.546386 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f5e0af25-d731-4633-99a6-fe2857ac21da tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.642689 Sep 30 09:46:02.554237 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 5/7] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:46:01 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:02.707553 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d523042a-17d1-480b-b15f-e6d7da3a2627 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:02.708777 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d523042a-17d1-480b-b15f-e6d7da3a2627 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:02.947828 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:46:02.948417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:46:02.948698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:46:02.951269 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:46:02.952444 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:46:03.025758 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:03.026254 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:03.029207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:03.030181 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:03.034208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:03.034774 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:03.120269 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Getting quotas for project b7f6b7ca2f3b4c20ab58940f7e24c1ed. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:46:03.140276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Getting quotas for user ae7915e1b2474d9384a0fdcb2ea177b6 and project b7f6b7ca2f3b4c20ab58940f7e24c1ed. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:46:03.172968 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:46:03.179489 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d523042a-17d1-480b-b15f-e6d7da3a2627 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:03.182070 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:46:03.189490 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d523042a-17d1-480b-b15f-e6d7da3a2627 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1222 microversion: 2.1 time: 0.487017 Sep 30 09:46:03.194321 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 6/8] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:02 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1222 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:03.471716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f562dba6-5768-46e8-b34d-89ea6e1bbbfd tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.330639 Sep 30 09:46:03.475711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 3/9] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:46:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 3338 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 30 09:46:03.487030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c4355827-096a-47ef-957a-33ca6dd37055 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:03.488040 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c4355827-096a-47ef-957a-33ca6dd37055 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:03.583550 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c4355827-096a-47ef-957a-33ca6dd37055 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:03.595029 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c4355827-096a-47ef-957a-33ca6dd37055 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1214 microversion: 2.1 time: 0.113110 Sep 30 09:46:03.597584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 7/10] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:03 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1214 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:04.234408 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:04.235098 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:04.265517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:04.267046 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:04.268210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:04.271341 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:04.272369 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:04.275007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:04.545812 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0d53ea5c-7296-4a71-9e0a-b7f46172cbff tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c1070ee0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:04.618976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:04.647373 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-92da4dcd-0f67-4418-b419-bf09643e0ac0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1218 microversion: 2.1 time: 0.428128 Sep 30 09:46:04.648736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 4/11] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:04 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1218 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:04.662386 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-36eda8ba-a9f6-43d3-821f-667178b19530 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:04.664402 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-36eda8ba-a9f6-43d3-821f-667178b19530 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:04.671110 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0d53ea5c-7296-4a71-9e0a-b7f46172cbff tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/e6659e1f-7ac4-414f-b8ea-7dcca61a977d" status: 200 len: 350 microversion: 2.1 time: 0.131720 Sep 30 09:46:04.672018 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 8/12] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:46:04 2022] GET /compute/v2.1/os-volumes/e6659e1f-7ac4-414f-b8ea-7dcca61a977d => generated 350 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:04.686574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-31d1fbd6-0d11-4261-98e4-dd7b14f9f0c0 tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c107d310>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:04.885159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-36eda8ba-a9f6-43d3-821f-667178b19530 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:04.907278 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-36eda8ba-a9f6-43d3-821f-667178b19530 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1214 microversion: 2.1 time: 0.250832 Sep 30 09:46:04.909723 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 5/13] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:04 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1214 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:04.943325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-31d1fbd6-0d11-4261-98e4-dd7b14f9f0c0 tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.261618 Sep 30 09:46:04.946242 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 9/14] 10.222.0.39 () {58 vars in 1134 bytes} [Fri Sep 30 09:46:04 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:04.987159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c57208cc-d1c1-46c6-b567-c5b711e90813 tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c10529d0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:05.176478 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c57208cc-d1c1-46c6-b567-c5b711e90813 tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.196363 Sep 30 09:46:05.177431 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 6/15] 10.222.0.39 () {58 vars in 1149 bytes} [Fri Sep 30 09:46:04 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:05.194812 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c5dac47d-671d-41d4-a632-25f54790fb5b tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c107dca0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:05.427743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c5dac47d-671d-41d4-a632-25f54790fb5b tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.239438 Sep 30 09:46:05.429110 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 10/16] 10.222.0.39 () {58 vars in 1134 bytes} [Fri Sep 30 09:46:05 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:05.444876 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ac0b0b89-97d7-4abf-a8e9-8feb3ebd0eca tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c0fd55e0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:05.673736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:05.674645 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:05.687057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:05.687563 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:05.688053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:05.688960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:05.689188 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:05.689614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:05.794005 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ac0b0b89-97d7-4abf-a8e9-8feb3ebd0eca tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.354184 Sep 30 09:46:05.795745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 7/17] 10.222.0.39 () {58 vars in 1167 bytes} [Fri Sep 30 09:46:05 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:05.810450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2afebf81-70cd-45fb-bb4f-3020ad270240 tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c0ffcca0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:06.074637 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2afebf81-70cd-45fb-bb4f-3020ad270240 tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.268764 Sep 30 09:46:06.086166 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 8/18] 10.222.0.39 () {58 vars in 1170 bytes} [Fri Sep 30 09:46:05 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:46:06.091723 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:06.092615 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:06.119049 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:06.119962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:06.122018 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:06.123532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:06.124469 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:06.125412 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:06.363186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:06.390323 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fc447e84-c13f-4677-aaaa-ca53aa31d518 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1274 microversion: 2.1 time: 0.720633 Sep 30 09:46:06.395844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 11/19] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:05 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1274 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:06.405196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a24c1d4c-e737-4fc5-bbe5-c462976c643a tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c0ff4e50>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:06.428960 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:06.445094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-bdcda6c8-2c2f-4837-b151-26657ff2326d tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1210 microversion: 2.1 time: 0.357535 Sep 30 09:46:06.446325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 9/20] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:06 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1210 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:06.602520 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a24c1d4c-e737-4fc5-bbe5-c462976c643a tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.205054 Sep 30 09:46:06.603142 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 12/21] 10.222.0.39 () {58 vars in 1155 bytes} [Fri Sep 30 09:46:06 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:06.629971 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-650f7258-b504-4c30-b564-5cea31085bce tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c0ffcee0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:06.938811 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-650f7258-b504-4c30-b564-5cea31085bce tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.318771 Sep 30 09:46:06.938811 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 10/22] 10.222.0.39 () {58 vars in 1188 bytes} [Fri Sep 30 09:46:06 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:06.950827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-913c2408-cdbf-4fd5-9764-f1ce7f48844c tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] Calling method '.version_select at 0x7f72c0fa15e0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:07.223351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-913c2408-cdbf-4fd5-9764-f1ce7f48844c tempest-VolumesTestJSON-1184215611 tempest-VolumesTestJSON-1184215611-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.276143 Sep 30 09:46:07.224158 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 13/23] 10.222.0.39 () {58 vars in 1155 bytes} [Fri Sep 30 09:46:06 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:07.424927 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:07.425581 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:07.458274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:07.459937 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:07.468657 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:07.470442 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:07.471039 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:07.471483 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:07.489401 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:07.490993 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:07.514128 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:07.518087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:07.524567 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:07.529732 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:07.529732 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:07.529732 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:07.654417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:07.673312 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-762b7e00-f7f6-46c5-ae2f-55834bd68218 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1292 microversion: 2.1 time: 0.256030 Sep 30 09:46:07.674481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 11/24] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:07 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1292 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:07.739361 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:07.755106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f317dcda-243c-401d-a83d-8689e9ad75d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1266 microversion: 2.1 time: 0.282536 Sep 30 09:46:07.756026 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 14/25] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:07 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1266 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:07.933962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1841590476", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d038a068-f643-497f-8e6f-7d6e8fb5892b"}], "security_groups": [{"name": "tempest-securitygroup--1581862540"}], "key_name": "tempest-keypair-2119459492", "user_data": "}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:08.542162 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] validate_networks() for [('d038a068-f643-497f-8e6f-7d6e8fb5892b', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:46:08.701777 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:08.708352 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:08.731903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:08.732608 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:08.733072 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:08.735792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:08.737506 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:08.738345 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:09.250256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:09.263992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ff47ab4e-7767-474f-8e82-a5c8b326c2c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1280 microversion: 2.1 time: 0.562454 Sep 30 09:46:09.271524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 15/26] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:08 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1280 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:09.292063 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:09.307202 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:09.326000 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:09.326790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:09.327899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:09.329808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:09.331046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:09.331614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:09.767275 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:09.782614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7c99409a-3a55-41ef-b1a2-67514a3297f6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1272 microversion: 2.1 time: 0.509870 Sep 30 09:46:09.785720 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 16/27] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:09 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1272 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:10.308294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:10.309262 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:10.318873 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:10.326833 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:10.327568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:10.328662 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:10.329394 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:10.330004 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:10.768838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=4 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:11.163459 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:11.173037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-10336987-4e18-479f-afb1-a54723c6e3a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.878143 Sep 30 09:46:11.179769 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 17/28] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:10 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 885 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:11.184854 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:11.187465 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:11.209739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:11.212649 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:11.213643 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:11.215092 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:11.215783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:11.216503 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:11.588126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:46:11.588727 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:46:11.589338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:46:11.589852 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:46:11.590470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:46:11.612113 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:11.618123 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:11.627056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:11.628654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:11.638363 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:11.638363 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:11.694426 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for project 86e6b0501cb94028be07bddd2182638b. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:46:11.734116 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for user 8c4ced79c3ff4842b30b472acb555cd5 and project 86e6b0501cb94028be07bddd2182638b. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:46:11.797165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:46:11.800801 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:46:11.951452 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:11.962257 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-be75e525-b459-48c9-b323-932a83d1c469 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.781041 Sep 30 09:46:11.963052 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 18/29] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:11 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:11.995218 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-28915d3c-22dd-4b3d-8f35-852eeaad9ccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 4.068199 Sep 30 09:46:11.999245 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 12/30] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:46:07 2022] POST /compute/v2.1/servers => generated 402 bytes in 4073 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:46:12.010745 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0582226f-5eee-451f-9a7b-b234feb899c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:12.011519 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0582226f-5eee-451f-9a7b-b234feb899c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:12.145818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0582226f-5eee-451f-9a7b-b234feb899c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:12.157947 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0582226f-5eee-451f-9a7b-b234feb899c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1242 microversion: 2.1 time: 0.151125 Sep 30 09:46:12.159354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 19/31] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:12 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1242 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:12.207821 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:12.207821 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:12.218404 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:12.218940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:12.225064 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:12.226174 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:12.227816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:12.229137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:12.479539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:12.489251 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-31ad3470-84f5-4622-9e71-f055398df523 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.292821 Sep 30 09:46:12.490104 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 13/32] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:12 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:12.962642 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-540b82ee-b081-4bbe-9e78-09e0e36972cf req-9b748d9c-d00d-438a-a2b7-0d4e08b398e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fd8e4eb8-1261-4abf-82ec-e1d8afda21ec", "tag": "45105b4f-28b5-44d6-bdd3-42f2c0b33de7"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:12.976002 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-540b82ee-b081-4bbe-9e78-09e0e36972cf req-9b748d9c-d00d-438a-a2b7-0d4e08b398e4 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:12.976508 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-540b82ee-b081-4bbe-9e78-09e0e36972cf req-9b748d9c-d00d-438a-a2b7-0d4e08b398e4 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:12.976931 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-540b82ee-b081-4bbe-9e78-09e0e36972cf req-9b748d9c-d00d-438a-a2b7-0d4e08b398e4 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:12.984981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:12.985762 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:12.991884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:12.992650 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:12.995169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:12.995169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:12.995169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:12.995169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:12.996005 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-540b82ee-b081-4bbe-9e78-09e0e36972cf req-9b748d9c-d00d-438a-a2b7-0d4e08b398e4 service nova] Creating event network-changed:45105b4f-28b5-44d6-bdd3-42f2c0b33de7 for instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec on hv-cinder-maste Sep 30 09:46:12.996716 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-540b82ee-b081-4bbe-9e78-09e0e36972cf req-9b748d9c-d00d-438a-a2b7-0d4e08b398e4 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=127456) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 30 09:46:12.997095 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-540b82ee-b081-4bbe-9e78-09e0e36972cf req-9b748d9c-d00d-438a-a2b7-0d4e08b398e4 service nova] Acquired lock "compute-rpcapi-router" {{(pid=127456) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 30 09:46:12.998278 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:12.998779 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:12.999192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:13.000045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:13.000483 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:13.000891 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:13.009256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.compute.rpcapi [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 30 09:46:13.010310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] Releasing lock "compute-rpcapi-router" {{(pid=127456) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 30 09:46:13.031589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bd50bd77-34e8-454b-b0e4-aed522246b7e None None] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.122557 Sep 30 09:46:13.032216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 20/33] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:46:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:13.131795 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:13.141430 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3f4b5760-504c-430c-8bd3-3f47fb6cbf1f tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.160319 Sep 30 09:46:13.142011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 14/34] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:12 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:13.186142 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dcd781ae-e347-4c8b-8b2c-060932e9ab93 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:13.186947 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dcd781ae-e347-4c8b-8b2c-060932e9ab93 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:13.247214 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dcd781ae-e347-4c8b-8b2c-060932e9ab93 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:13.254949 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dcd781ae-e347-4c8b-8b2c-060932e9ab93 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1242 microversion: 2.1 time: 0.073901 Sep 30 09:46:13.255518 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 21/35] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:13 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1242 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:13.513262 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:13.513868 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:13.521728 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:13.522263 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:13.522770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:13.523417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:13.523872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:13.524542 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:13.732675 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:13.741442 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-69f268ca-ec1e-4a7a-8651-da829d6755d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.231972 Sep 30 09:46:13.741945 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 15/36] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:13 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:14.165115 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:14.170995 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:14.173060 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:14.173550 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:14.173994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:14.174719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:14.175390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:14.175994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:14.282154 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:14.283565 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:14.291222 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:14.291721 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:14.292160 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:14.293231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:14.293880 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:14.294498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:14.346902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:14.365736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5f3fb6c7-6c4f-4378-a3a9-ce56c9737538 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.204527 Sep 30 09:46:14.381433 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 22/37] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:14 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:14.404180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:14.410900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-201bacee-de2f-47a0-960d-d698cd051f90 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1246 microversion: 2.1 time: 0.136737 Sep 30 09:46:14.411551 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 16/38] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:14 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1246 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:14.767569 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:14.768183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:14.778091 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:14.779212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:14.781140 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:14.783589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:14.785348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:14.788670 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:15.018904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:15.032621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-78aeed11-7e8c-47ba-a0ea-c6f5540d6140 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.269763 Sep 30 09:46:15.042473 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 23/39] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:14 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:15.389867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:15.391232 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:15.397659 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:15.398183 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:15.398832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:15.399953 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:15.401026 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:15.401455 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:15.437411 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:15.444223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:15.449496 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:15.450294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:15.450959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:15.459250 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:15.459790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:15.460289 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:15.732071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:15.768479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-96d54c1d-58b9-4f92-9e36-e3525f7bc574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1246 microversion: 2.1 time: 0.337332 Sep 30 09:46:15.787199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 24/40] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:15 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1246 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:15.864920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:15.874169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ee684cb0-cad6-4570-ae2f-436a4fbfd3cb tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.488014 Sep 30 09:46:15.885643 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 17/41] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:15 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:16.064240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:16.064240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:16.068920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:16.069628 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:16.070062 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:16.071218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:16.071721 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:16.072184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:16.476468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:16.494672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5f0d77a7-0461-4ce1-9d74-05b268b7a57e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.437603 Sep 30 09:46:16.497124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 25/42] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:16 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:16.801444 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:16.802279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:16.809430 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:16.810170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:16.811390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:16.812281 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:16.812926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:16.813496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:16.877010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:16.884593 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1a8c31a4-2eee-4b4c-92c7-e63579dafa00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1246 microversion: 2.1 time: 0.087360 Sep 30 09:46:16.885256 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 18/43] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:16 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1246 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:16.899904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:16.900565 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:16.908112 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:16.908606 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:16.909040 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:16.909730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:16.910216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:16.910680 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:17.046505 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:17.054193 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-43ca0cdb-fcb3-4116-ab8d-aba7e11767b4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.158081 Sep 30 09:46:17.054791 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 26/44] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:16 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:17.560159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:17.561689 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:17.575253 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:17.580351 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:17.583527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:17.585491 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:17.587154 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:17.587955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:17.866450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:17.881021 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8026569a-c2bc-47a5-91fe-3913c5b903b7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.324752 Sep 30 09:46:17.881813 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 19/45] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:17 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:17.907291 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:17.907976 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:17.915250 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:17.919963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:17.919963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:17.919963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:17.919963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:17.920289 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:17.960084 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-aad2008a-a6a0-43de-bc97-efdc8e8b87f8 req-8506f8e1-00bc-4e6c-a421-58f8acafda39 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d2701f34-7f9f-49cb-a2c3-9f84c1a96e31", "tag": "d3f7eb0a-905a-4efa-b08a-3fca026724aa"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:17.992494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-aad2008a-a6a0-43de-bc97-efdc8e8b87f8 req-8506f8e1-00bc-4e6c-a421-58f8acafda39 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:17.992976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-aad2008a-a6a0-43de-bc97-efdc8e8b87f8 req-8506f8e1-00bc-4e6c-a421-58f8acafda39 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:17.993401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-aad2008a-a6a0-43de-bc97-efdc8e8b87f8 req-8506f8e1-00bc-4e6c-a421-58f8acafda39 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:18.016784 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:18.028808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-16289965-6d43-4e07-aba8-000c2a044b16 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1246 microversion: 2.1 time: 0.125450 Sep 30 09:46:18.029820 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 27/46] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:17 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1246 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:18.035245 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-aad2008a-a6a0-43de-bc97-efdc8e8b87f8 req-8506f8e1-00bc-4e6c-a421-58f8acafda39 service nova] Creating event network-changed:d3f7eb0a-905a-4efa-b08a-3fca026724aa for instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 on hv-cinder-maste Sep 30 09:46:18.035998 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-aad2008a-a6a0-43de-bc97-efdc8e8b87f8 req-8506f8e1-00bc-4e6c-a421-58f8acafda39 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=127455) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 30 09:46:18.036874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-aad2008a-a6a0-43de-bc97-efdc8e8b87f8 req-8506f8e1-00bc-4e6c-a421-58f8acafda39 service nova] Acquired lock "compute-rpcapi-router" {{(pid=127455) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 30 09:46:18.038022 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:18.038964 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:18.039807 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:18.040884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:18.041623 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:18.042394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:18.057874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.compute.rpcapi [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 30 09:46:18.058979 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] Releasing lock "compute-rpcapi-router" {{(pid=127455) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 30 09:46:18.078712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:18.079273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:18.091360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:18.092095 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:18.093613 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:18.095401 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:18.097075 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:18.101237 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:18.105642 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-71560aab-4260-4ea4-97a6-a87412df6e00 None None] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.147473 Sep 30 09:46:18.107779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 20/47] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:46:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:46:18.369703 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:18.380634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-54036781-e51d-4af8-9e05-ac213088d309 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.307150 Sep 30 09:46:18.381218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 28/48] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:18 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:18.555308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-3e0d8d7e-4b7e-4c3e-84d3-614ae104fc6d req-5704cef7-9e5d-410f-abbc-769a1f637db7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2a244aaa-c28f-405e-ad45-39386205bb41", "tag": "8588e984-7119-40f8-9e2a-800c04c4c6aa"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:18.566029 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3e0d8d7e-4b7e-4c3e-84d3-614ae104fc6d req-5704cef7-9e5d-410f-abbc-769a1f637db7 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:18.575667 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3e0d8d7e-4b7e-4c3e-84d3-614ae104fc6d req-5704cef7-9e5d-410f-abbc-769a1f637db7 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:18.576115 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3e0d8d7e-4b7e-4c3e-84d3-614ae104fc6d req-5704cef7-9e5d-410f-abbc-769a1f637db7 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:18.588442 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-3e0d8d7e-4b7e-4c3e-84d3-614ae104fc6d req-5704cef7-9e5d-410f-abbc-769a1f637db7 service nova] Creating event network-changed:8588e984-7119-40f8-9e2a-800c04c4c6aa for instance 2a244aaa-c28f-405e-ad45-39386205bb41 on hv-cinder-maste Sep 30 09:46:18.593880 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-3e0d8d7e-4b7e-4c3e-84d3-614ae104fc6d req-5704cef7-9e5d-410f-abbc-769a1f637db7 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.293067 Sep 30 09:46:18.594483 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 21/49] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:46:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:18.903704 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:18.904612 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:18.910572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:18.911696 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:18.912123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:18.912804 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:18.913396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:18.913809 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:19.053229 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:19.053961 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:19.071190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:19.074831 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:19.074831 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:19.074831 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:19.075172 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:19.075922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:19.089102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:19.102255 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-81b3534c-abbb-4a65-bf2c-8911dce19729 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.202210 Sep 30 09:46:19.114837 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 29/50] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:18 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:19.239755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:19.247961 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8decc849-1fb0-41e1-be06-6983f6150ca5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1246 microversion: 2.1 time: 0.199985 Sep 30 09:46:19.249758 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 22/51] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:19 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1246 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:19.403222 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:19.404152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:19.412012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:19.412514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:19.413192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:19.413918 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:19.421306 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:19.422075 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:19.698247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:19.717434 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-20b62e2e-cdef-481c-a95c-f725d41e54c9 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.318084 Sep 30 09:46:19.720951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 30/52] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:19 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:20.128833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:20.129500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:20.139629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:20.140141 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:20.140567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:20.141273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:20.147072 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:20.148367 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:20.280624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:20.281238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:20.308795 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:20.309636 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:20.315263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:20.315263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:20.315263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:20.315263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:20.396517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:20.403784 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=100 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:20.423288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e6fc8d4f-6226-4efe-b276-b88eda331fd6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.299166 Sep 30 09:46:20.423889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 23/53] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:20 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:20.598244 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:20.634923 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-87a1e70e-c146-4fdf-a1ed-9169a57da68a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1238 microversion: 2.1 time: 0.363963 Sep 30 09:46:20.637615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 31/54] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:20 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1238 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:20.743683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:20.744295 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:20.776652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:20.782535 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:20.796895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:20.796895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:20.796895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:20.796895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:21.455969 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:21.463585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:21.479754 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:21.483278 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:21.485022 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:21.485461 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:21.486176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:21.486650 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:21.487072 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:21.511258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:21.523588 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0d1380d9-d45f-4493-b0f6-e2be646b27cc tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.783454 Sep 30 09:46:21.525120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 24/55] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:20 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:21.668440 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:21.669040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:21.675895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:21.676377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:21.680819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:21.680819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:21.680819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:21.680819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:21.767112 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=16 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:21.801491 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:21.812940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-17ad94a0-8802-4139-8b8f-4f214a42c426 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1294 microversion: 2.1 time: 0.148426 Sep 30 09:46:21.815788 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 25/56] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:21 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1294 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:22.087336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:22.100462 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8a2e5d0a-a56a-42e8-8975-f3e8bd4b49ca tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.658827 Sep 30 09:46:22.104170 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 32/57] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:21 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:22.551428 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:22.552737 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:22.559492 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:22.560196 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:22.560970 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:22.566221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:22.566221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:22.566221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:22.751930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:22.756021 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:22.763359 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d013efb8-5fd7-4109-aa10-d42f9df02c41 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.214255 Sep 30 09:46:22.764959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 26/58] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:22 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:22.845075 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:22.845566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:22.854704 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:22.859192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:22.859631 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:22.860349 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:22.860797 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:22.861212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:23.007372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:23.112102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:23.132705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-cb45a0ca-f233-415d-8698-8e7be2d987c0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1312 microversion: 2.1 time: 0.286942 Sep 30 09:46:23.132705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 33/59] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:22 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1312 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:23.133103 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:23.134009 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:23.161247 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:23.161247 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:23.161247 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:23.164180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:23.164659 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:23.172429 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:23.797114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:23.799786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:23.807077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=26 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:23.823738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:23.825423 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:23.829118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:23.830092 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:23.830960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:23.832142 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:23.879178 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:23.892540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a20eaee2-24a8-4aad-8232-ea4e833644df tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.764732 Sep 30 09:46:23.894288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 27/60] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:23 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 767 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:24.108956 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:24.117106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ad6de6e3-6cf2-4990-ae1f-9c9296ad3e88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.329927 Sep 30 09:46:24.117669 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 34/61] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:23 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:24.153532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:24.154698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:24.160587 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:24.161056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:24.161480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:24.162255 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:24.162730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:24.163144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:24.226779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:24.236632 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0b3c4411-5a62-4761-b153-7c4e3a039d17 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1300 microversion: 2.1 time: 0.087046 Sep 30 09:46:24.237221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 28/62] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:24 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1300 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:24.919576 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:24.920527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:24.926831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:24.927285 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:24.927727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:24.928477 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:24.928832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:24.929463 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:25.141711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:25.142322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:25.159790 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:25.166282 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:25.167233 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:25.168163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:25.168812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:25.169234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:25.171364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:25.191663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-913e83ed-ed52-49ca-b801-f3b26ae8f614 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.274749 Sep 30 09:46:25.203033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 35/63] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:24 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:25.266216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:25.267355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:25.286914 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:25.287740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:25.290800 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:25.291686 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:25.293893 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:25.294520 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:25.401949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:25.416355 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1c5de419-2d14-4119-bfdc-4eca5d546f64 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.279838 Sep 30 09:46:25.417159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 29/64] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:25 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:25.486119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:25.495152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0790c7f0-cc51-42bf-992d-a1785d550fbd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.233828 Sep 30 09:46:25.496410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 36/65] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:25 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:26.225708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:26.226318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:26.234032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:26.234746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:26.235205 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:26.238939 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:26.239476 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:26.239930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:26.424513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:26.432217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4fe41b55-e4b2-4190-b232-d6fba9ddf5c2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.210393 Sep 30 09:46:26.432788 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 30/66] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:26 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:26.438543 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:26.439250 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:26.445039 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:26.445515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:26.445938 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:26.446702 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:26.447280 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:26.447752 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:26.519980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:26.523449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:26.545660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:26.546152 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:26.546580 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:26.547320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:26.547767 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:26.548176 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:26.687244 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:26.695896 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-04ab11e4-85f9-4d0c-8881-e3a8d917cf6e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.261101 Sep 30 09:46:26.698765 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 37/67] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:26 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:26.734533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:26.742435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b8df0850-a923-4a82-950e-e062aa9cb286 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.226426 Sep 30 09:46:26.743043 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 31/68] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:26 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:27.456684 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:27.458409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:27.468140 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:27.468634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:27.470126 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:27.470873 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:27.471326 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:27.471747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:27.600949 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:27.609240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0b4088dd-1b31-4ed2-bb90-6fbefce1392f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.157174 Sep 30 09:46:27.611310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 38/69] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:27 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:27.725133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:27.725734 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:27.731554 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:27.732661 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:27.733087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:27.733765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:27.734208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:27.734643 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:27.766178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:27.766926 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:27.773783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:27.774385 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:27.775170 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:27.775736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:27.776282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:27.776808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:27.943525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:27.955388 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-71d577d6-0dcf-49b5-a5aa-d2d60181a32e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.233894 Sep 30 09:46:27.956258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 32/70] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:27 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:27.972348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:27.979931 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bfd45495-0da8-4103-a923-f13e9b6daa8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.218075 Sep 30 09:46:27.980502 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 39/71] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:27 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:28.637231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:28.637829 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:28.639850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=6 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:28.644658 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:28.645139 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:28.645737 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:28.646425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:28.647091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:28.647663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:28.784600 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:28.792500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a928555e-6e96-4837-9961-12d3ce19da6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.159075 Sep 30 09:46:28.793060 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 33/72] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:28 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:28.978990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:28.984338 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:28.985835 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:28.986322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:28.986794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:28.987493 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:28.987948 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:28.988369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:29.003415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:29.004232 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:29.010624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:29.011520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:29.012039 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:29.012827 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:29.013875 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:29.014386 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:29.216386 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:29.220919 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:29.225802 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-48527c89-4641-4f2e-8677-7fb6b8036212 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.226455 Sep 30 09:46:29.226987 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 34/73] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:28 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:29.230659 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-20c14a30-b3da-4514-9892-72ce33c98a87 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.255466 Sep 30 09:46:29.231503 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 40/74] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:28 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:29.815640 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:29.816291 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:29.819161 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=6 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:29.822506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:29.823003 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:29.823426 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:29.824100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:29.824542 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:29.824949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:29.957396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:29.965245 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0b3c5340-37a6-4531-92ea-293ebd0daeee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.153476 Sep 30 09:46:29.965810 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 35/75] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:29 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:30.248068 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:30.248680 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:30.252929 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:30.253679 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:30.254761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:30.255250 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:30.255684 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:30.256372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:30.256826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:30.257249 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:30.260046 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:30.260525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:30.260952 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:30.261799 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:30.262253 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:30.262711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:30.460021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:30.464728 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:30.472585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ddd7785a-090d-4cb2-9a6d-3400c6a14925 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.228513 Sep 30 09:46:30.473251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 41/76] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:30 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:30.483549 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5770a9f7-439f-43aa-94dd-b9166c1ac588 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.225091 Sep 30 09:46:30.483549 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 36/77] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:30 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:30.989590 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:30.990228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:30.996263 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:30.996735 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:30.997168 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:30.998406 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:30.998877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:30.999336 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:31.204126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:31.212449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c704f336-1d54-400b-9b92-d9c51b515f33 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.227642 Sep 30 09:46:31.213139 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 37/78] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:30 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:31.495477 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:31.496485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:31.504693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:31.505299 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:31.511389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:31.511576 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:31.511883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:31.512196 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:31.512315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:31.512760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:31.513171 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:31.513458 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:31.513699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:31.514018 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:31.514447 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:31.514540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:31.727115 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:31.733400 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:31.735314 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ba2cd2e2-8b71-4909-8267-df9901695548 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.244032 Sep 30 09:46:31.735884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 42/79] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:31 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:31.740858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8044a1f4-e29c-4bd6-b1c5-17bde8ef7a8b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.240003 Sep 30 09:46:31.741419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 38/80] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:31 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:31.796864 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-ced3f1e1-c69a-4677-ae9d-122a19c4323a req-d2730481-63db-4a56-82ec-9e550f9f7956 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0bc1e701-2247-4837-81c0-289806279c31", "tag": "cc523046-219f-42b7-a7b9-700936140c81"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:31.806657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ced3f1e1-c69a-4677-ae9d-122a19c4323a req-d2730481-63db-4a56-82ec-9e550f9f7956 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:31.809763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ced3f1e1-c69a-4677-ae9d-122a19c4323a req-d2730481-63db-4a56-82ec-9e550f9f7956 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:31.810252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ced3f1e1-c69a-4677-ae9d-122a19c4323a req-d2730481-63db-4a56-82ec-9e550f9f7956 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:31.826066 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-ced3f1e1-c69a-4677-ae9d-122a19c4323a req-d2730481-63db-4a56-82ec-9e550f9f7956 service nova] Creating event network-changed:cc523046-219f-42b7-a7b9-700936140c81 for instance 0bc1e701-2247-4837-81c0-289806279c31 on hv-cinder-maste Sep 30 09:46:31.829949 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-ced3f1e1-c69a-4677-ae9d-122a19c4323a req-d2730481-63db-4a56-82ec-9e550f9f7956 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036640 Sep 30 09:46:31.830634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 43/81] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:46:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:32.235567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:32.236176 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:32.242001 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:32.247808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:32.247808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:32.247808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:32.247808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:32.248517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:32.370199 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:32.378516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8182401d-e819-4546-a0a6-67042b348e27 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.146873 Sep 30 09:46:32.379101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 39/82] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:32 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:32.760158 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:32.760855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:32.764835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:32.765445 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:32.770410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:32.770410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:32.770410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:32.770410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:32.770733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:32.770733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:32.771964 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:32.772440 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:32.772870 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:32.773578 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:32.774070 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:32.774500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:32.963764 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:32.968838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:32.972173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-35f2d145-14f5-401f-a86c-f3292d42c54e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.216689 Sep 30 09:46:32.977569 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2b5e151c-27fe-49c3-8358-9cfa595fed03 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.216474 Sep 30 09:46:32.978562 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 40/83] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:32 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:32.986845 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 44/84] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:32 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:33.400717 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:33.401389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:33.408143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:33.408616 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:33.409041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:33.409719 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:33.410170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:33.410582 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:33.542996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:33.551451 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-729fd4a5-4a39-4710-8944-cb4dba927827 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.154649 Sep 30 09:46:33.552102 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 41/85] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:33 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:33.994807 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:33.995410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:34.002163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:34.002319 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:34.002770 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:34.002883 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:34.003199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:34.003876 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:34.004355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:34.004804 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:34.008982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:34.009462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:34.009901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:34.010613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:34.011085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:34.011505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:34.194266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:34.201191 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:34.202434 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-6eb8d3b2-739c-40d6-a838-0053b52896b9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.211462 Sep 30 09:46:34.203037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 45/86] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:33 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:34.209384 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-dc3cb052-c851-4880-96d0-0f7a917b18db tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.211110 Sep 30 09:46:34.209941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 42/87] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:33 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:34.574882 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:34.575485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:34.581319 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:34.581791 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:34.582216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:34.582912 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:34.583356 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:34.583769 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:34.716890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:34.724383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8087f3a1-36d2-4cc6-8ab9-0569bcd95f3a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.153523 Sep 30 09:46:34.725041 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 46/88] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:34 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:35.224776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:35.225391 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:35.231456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:35.231890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:35.232069 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:35.232354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:35.232498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:35.233314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:35.233765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:35.234190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:35.238543 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:35.239119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:35.239616 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:35.240364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:35.240794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:35.241213 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:35.254885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:35.433928 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:35.441435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:35.442106 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-19effcdf-1d22-4d12-85e9-0ed201289cae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.221107 Sep 30 09:46:35.442702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 43/89] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:35 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:35.449755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-18d0489d-3cad-4bf0-92cc-c7909aec5bb7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.221968 Sep 30 09:46:35.451253 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 47/90] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:35 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:35.746717 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:35.747862 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:35.753619 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:35.754090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:35.754510 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:35.755212 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:35.755655 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:35.756064 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:35.882163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:35.889970 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c5e761ff-80eb-44d7-83a0-3e8d87096c71 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.147357 Sep 30 09:46:35.890518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 44/91] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:35 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:36.464924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:36.465531 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:36.472409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:36.472895 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:36.473031 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:36.473448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:36.473585 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:36.474010 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:36.474464 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:36.474913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:36.479402 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:36.479866 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:36.480293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:36.480970 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:36.481409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:36.481842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:36.493089 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=100 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:36.670634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:36.672216 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:36.678202 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f92a105a-870a-4a79-a483-4b8b1c61524c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.217168 Sep 30 09:46:36.678717 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 48/92] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:36 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:36.680241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ad55f5fc-54b2-4a26-bee2-79e86b2fd53b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.211035 Sep 30 09:46:36.680927 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 45/93] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:36 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:36.912734 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:36.913339 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:36.919155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:36.919624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:36.920049 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:36.920721 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:36.921167 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:36.921582 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:37.190121 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:37.198044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-13de3fd3-8474-45ea-9033-1f9d17fa0f49 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.289113 Sep 30 09:46:37.198821 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 49/94] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:36 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:37.704033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:37.704641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:37.707594 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:37.708201 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:37.714831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:37.716309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:37.716878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:37.717439 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:37.717562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:37.717966 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:37.718068 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:37.718347 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:37.718459 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:37.719055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:37.719502 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:37.719911 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:37.903651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:37.911112 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d8e53b06-5f91-4559-9406-3eb65951fd88 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.211471 Sep 30 09:46:37.911676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 50/95] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:37 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:37.917394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:37.925031 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-472b006d-ed84-408f-b900-5e79cfd76d2a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.227931 Sep 30 09:46:37.925666 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 46/96] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:37 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:38.222208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:38.222839 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:38.228741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:38.229211 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:38.229658 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:38.230330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:38.230809 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:38.231223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:38.363267 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:38.371040 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-87cadd3c-e2ab-47e6-8e77-83ea683eb8f0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.152447 Sep 30 09:46:38.371597 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 51/97] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:38 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:38.933173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:38.933782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:38.939663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:38.940148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:38.940591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:38.941311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:38.941764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:38.942274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:38.956814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:38.956814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:38.956814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:38.956814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:38.957443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:38.957443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:38.957443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:38.957443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:39.139134 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:39.143247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:39.146868 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7909967e-1e66-4aee-aeaa-cda3e470d8d0 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.217449 Sep 30 09:46:39.147448 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 47/98] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:38 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:39.150895 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9ad94a89-0d4b-4a9a-973c-de691377fbc4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.207304 Sep 30 09:46:39.151450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 52/99] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:38 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:39.394208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:39.394867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:39.400678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:39.401158 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:39.401590 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:39.402274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:39.402753 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:39.403844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:39.530724 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:39.538152 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3a8fd25d-503c-4a2b-9162-5be1367964a6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.147790 Sep 30 09:46:39.538734 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 48/100] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:39 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:40.170340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:40.171079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:40.177051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:40.177610 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:40.178110 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:40.178826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:40.179275 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:40.179696 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:40.187420 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:40.188028 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:40.193915 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:40.194392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:40.194847 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:40.195695 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:40.196144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:40.196555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:40.365249 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:40.373004 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1ca42630-0362-4ec8-b2d7-ca9270f26654 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.207408 Sep 30 09:46:40.373578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 53/101] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:40 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:40.385828 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:40.393580 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-65d76f8e-ce20-424e-a14c-1803b0b08a08 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.223854 Sep 30 09:46:40.394142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 49/102] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:40 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:40.560829 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:40.561419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:40.567284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:40.567750 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:40.568171 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:40.568857 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:40.569286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:40.569693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:40.780602 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:40.790655 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-541e8797-c15b-4436-8f66-f8138f95bb84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.233662 Sep 30 09:46:40.791734 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 54/103] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:40 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:41.396293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:41.396795 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:41.403231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:41.403948 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:41.404376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:41.405073 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:41.405517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:41.405955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:41.416595 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:41.417883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:41.424252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:41.425002 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:41.425635 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:41.426536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:41.427160 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:41.427854 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:41.635922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:41.637532 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:41.645069 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-aa2182d0-6db0-4c92-b1c1-8b73528360e4 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.252913 Sep 30 09:46:41.645261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-55f44fea-0b65-4978-96f4-94ad25d1141c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.232643 Sep 30 09:46:41.646017 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 50/104] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:41 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:41.646200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 55/105] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:41 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:41.817358 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:41.818218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:41.825250 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:41.825727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:41.826149 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:41.826867 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:41.827312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:41.827724 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:41.965971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:41.973510 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ce9a1f03-85ca-4a3f-b648-653b108b914f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.163480 Sep 30 09:46:41.974861 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 56/106] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:41 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:42.674260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:42.676459 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:42.685409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:42.686138 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:42.686722 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:42.687541 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:42.692747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:42.692747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:42.717392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-7546fb07-e0de-43e9-8708-8dd0836671e3 req-de06db16-2165-4a85-b0f7-294dc62fb587 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d2701f34-7f9f-49cb-a2c3-9f84c1a96e31", "name": "network-vif-plugged", "status": "completed", "tag": "d3f7eb0a-905a-4efa-b08a-3fca026724aa"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:42.728392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-7546fb07-e0de-43e9-8708-8dd0836671e3 req-de06db16-2165-4a85-b0f7-294dc62fb587 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:42.728871 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-7546fb07-e0de-43e9-8708-8dd0836671e3 req-de06db16-2165-4a85-b0f7-294dc62fb587 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:42.729296 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-7546fb07-e0de-43e9-8708-8dd0836671e3 req-de06db16-2165-4a85-b0f7-294dc62fb587 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:42.741866 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-7546fb07-e0de-43e9-8708-8dd0836671e3 req-de06db16-2165-4a85-b0f7-294dc62fb587 service nova] Creating event network-vif-plugged:d3f7eb0a-905a-4efa-b08a-3fca026724aa for instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 on hv-cinder-maste Sep 30 09:46:42.752288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-7546fb07-e0de-43e9-8708-8dd0836671e3 req-de06db16-2165-4a85-b0f7-294dc62fb587 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.116665 Sep 30 09:46:42.753434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 51/107] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:46:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:42.765686 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:42.768280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:42.777297 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:42.778765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:42.778765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:42.779277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:42.779894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:42.780479 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:42.930521 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:42.942052 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fb7c9287-b711-4286-aeef-4b1747c9f985 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.277560 Sep 30 09:46:42.954733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 57/108] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:42 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:42.998609 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:42.999627 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:43.001543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:43.005673 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:43.006150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:43.006570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:43.007273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:43.007712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:43.008125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:43.011348 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9403b85e-ad96-4292-8259-4a647250af59 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.253521 Sep 30 09:46:43.011987 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 52/109] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:42 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:43.238020 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:43.245963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4e2299e3-5ba4-4677-8f37-23b833865ae3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.252280 Sep 30 09:46:43.247128 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 58/110] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:42 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:43.967750 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:43.968282 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:43.974319 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:43.974850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:43.975314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:43.976007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:43.976454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:43.976869 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:44.034685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:44.035307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:44.041526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:44.042016 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:44.042448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:44.043175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:44.043685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:44.044106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:44.156230 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:44.165792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3b8a4e94-9ba4-412c-a69e-9d9dcdc09927 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.202534 Sep 30 09:46:44.178839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 53/111] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:43 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:44.216468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:44.224161 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-04e6dbd3-277d-45cd-aa75-ab6bf1cdc5f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.193933 Sep 30 09:46:44.224727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 59/112] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:44 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:44.269410 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:44.270011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:44.275990 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:44.276322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:44.276745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:44.277422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:44.277895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:44.278314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:44.408094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:44.415571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-87c9d654-bf63-4c9b-ae3c-cfda40b94af5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.150007 Sep 30 09:46:44.416127 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 54/113] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:44 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:44.767737 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-cc4f6ac5-85e3-4a86-8315-a2590cfd3cec req-4db4a939-742e-4dbe-839f-6f2ea77f1db5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd8e4eb8-1261-4abf-82ec-e1d8afda21ec", "name": "network-vif-plugged", "status": "completed", "tag": "45105b4f-28b5-44d6-bdd3-42f2c0b33de7"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:44.779050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-cc4f6ac5-85e3-4a86-8315-a2590cfd3cec req-4db4a939-742e-4dbe-839f-6f2ea77f1db5 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:44.779050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-cc4f6ac5-85e3-4a86-8315-a2590cfd3cec req-4db4a939-742e-4dbe-839f-6f2ea77f1db5 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:44.779050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-cc4f6ac5-85e3-4a86-8315-a2590cfd3cec req-4db4a939-742e-4dbe-839f-6f2ea77f1db5 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:44.794492 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-cc4f6ac5-85e3-4a86-8315-a2590cfd3cec req-4db4a939-742e-4dbe-839f-6f2ea77f1db5 service nova] Creating event network-vif-plugged:45105b4f-28b5-44d6-bdd3-42f2c0b33de7 for instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec on hv-cinder-maste Sep 30 09:46:44.796771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-cc4f6ac5-85e3-4a86-8315-a2590cfd3cec req-4db4a939-742e-4dbe-839f-6f2ea77f1db5 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033267 Sep 30 09:46:44.797545 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 60/114] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:46:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:45.189263 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:45.189862 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:45.197062 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:45.198157 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:45.198602 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:45.199308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:45.200793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:45.201419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:45.247612 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:45.248297 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:45.254382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:45.254887 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:45.255317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:45.255983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:45.256435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:45.256845 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:45.381873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:45.390684 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ca837907-393b-420a-b46d-f486d44600ee tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.205414 Sep 30 09:46:45.391348 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 55/115] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:45 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:45.418890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:45.426366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b4c53e2c-4e7f-49d6-be14-8d02d075319a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.183464 Sep 30 09:46:45.426961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 61/116] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:45 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:45.437334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:45.438040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:45.444031 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:45.444509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:45.444976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:45.445770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:45.446219 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:45.446659 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:45.578523 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:45.586096 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-822d53f5-f205-4a66-b508-ef6853859775 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.152684 Sep 30 09:46:45.586678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 56/117] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:45 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:46.414262 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:46.414885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:46.421509 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:46.421981 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:46.422974 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:46.423156 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:46.423601 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:46.424046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:46.449076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:46.449803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:46.456037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:46.456541 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:46.456964 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:46.457704 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:46.458147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:46.458724 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:46.600175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:46.610485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-39923366-326e-4d57-bfab-e7338054922e tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.199932 Sep 30 09:46:46.611112 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 62/118] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:46 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:46.618375 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:46.619191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:46.626913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:46.626913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:46.627174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:46.627890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:46.628351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:46.628774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:46.630411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:46.639281 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7b56d4df-2580-45cd-8c7e-d8d0a0522878 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.193916 Sep 30 09:46:46.640459 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 57/119] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:46 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:46.774920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:46.783243 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-803ddeda-78d5-482e-9c1c-10da8fed9b9f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.171174 Sep 30 09:46:46.783790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 63/120] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:46 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:47.633595 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:47.634158 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:47.640043 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:47.640521 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:47.640944 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:47.641803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:47.642250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:47.642691 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:47.662852 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:47.664880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:47.669619 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:47.670099 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:47.671080 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:47.671882 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:47.672388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:47.672809 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:47.927554 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:47.927904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:47.927904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7e192c58-bf25-40b9-9fca-ea6d5256bdb4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.242876 Sep 30 09:46:47.927904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 64/121] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:47 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:47.927904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:47.928246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:47.928334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3f942457-7d1c-4b62-8670-44e30f560cec tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.284664 Sep 30 09:46:47.934516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 58/122] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:47 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:47.937471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:47.938216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:47.938748 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:47.939527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:47.939979 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:47.940404 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:48.094067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:48.094067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=100 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:48.094067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-76133a7d-8a0a-4682-83a8-3f018d15d68c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1322 microversion: 2.1 time: 0.185475 Sep 30 09:46:48.094067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 65/123] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:47 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1322 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:48.937711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:48.938955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:48.939181 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:48.939796 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:48.945956 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:48.949444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:48.949838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:48.949944 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:48.950040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:48.950131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:48.950220 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:48.950304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:48.950395 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:48.950481 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:48.950564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:48.950668 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:49.210907 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:49.218552 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e7a4c80e-34d5-47f8-a0f2-7303b5f51226 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.284951 Sep 30 09:46:49.230625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 59/124] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:48 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:49.230625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:49.230625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:49.230625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:49.231554 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:49.231554 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:49.231926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:49.232589 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:49.233747 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:49.249470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:49.265346 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f006a594-3b96-4c66-a40a-d1527935bdc7 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.330456 Sep 30 09:46:49.267846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 66/125] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:48 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:49.275246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-7d65cac9-412a-4782-b31c-30bb3a85f0be req-c3d9b4cb-8b66-42e8-95d2-a86f968424c4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2a244aaa-c28f-405e-ad45-39386205bb41", "name": "network-vif-plugged", "status": "completed", "tag": "8588e984-7119-40f8-9e2a-800c04c4c6aa"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:49.289443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7d65cac9-412a-4782-b31c-30bb3a85f0be req-c3d9b4cb-8b66-42e8-95d2-a86f968424c4 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:49.290148 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7d65cac9-412a-4782-b31c-30bb3a85f0be req-c3d9b4cb-8b66-42e8-95d2-a86f968424c4 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:49.292673 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7d65cac9-412a-4782-b31c-30bb3a85f0be req-c3d9b4cb-8b66-42e8-95d2-a86f968424c4 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:49.305299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-7d65cac9-412a-4782-b31c-30bb3a85f0be req-c3d9b4cb-8b66-42e8-95d2-a86f968424c4 service nova] Creating event network-vif-plugged:8588e984-7119-40f8-9e2a-800c04c4c6aa for instance 2a244aaa-c28f-405e-ad45-39386205bb41 on hv-cinder-maste Sep 30 09:46:49.311319 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-7d65cac9-412a-4782-b31c-30bb3a85f0be req-c3d9b4cb-8b66-42e8-95d2-a86f968424c4 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041887 Sep 30 09:46:49.312507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 67/126] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:46:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:49.605250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:49.610649 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:49.623128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7593c0e9-634c-4d86-8c43-f41a83d9a96e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1516 microversion: 2.1 time: 0.402487 Sep 30 09:46:49.624374 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 60/127] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:49 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1516 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:49.648632 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:49.651995 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:49.658055 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:49.658532 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:49.658983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:49.659710 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:49.660159 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:49.660588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:49.722858 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lazy-loading 'flavor' on Instance uuid d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:46:50.098319 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b8bde006-628b-42b9-8fb5-ae768a92d869 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31/action" status: 200 len: 32 microversion: 2.1 time: 0.454101 Sep 30 09:46:50.099249 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 68/128] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:46:49 2022] POST /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31/action => generated 32 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 09:46:50.110273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:50.110919 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:50.118574 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:50.119278 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:50.119856 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:50.120949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:50.121537 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:50.122098 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:50.269739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:50.273680 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:50.274364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:50.281808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2464277a-8301-4cbf-b125-f7c5fdcc4675 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.177154 Sep 30 09:46:50.283012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 61/129] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:50 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:50.289958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:50.290794 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:50.299843 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:50.300326 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:50.300903 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:50.301897 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:50.302490 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:50.303066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:50.314650 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:50.315333 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:50.316782 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:50.317714 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:50.320450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:50.320965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:50.538341 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:50.547426 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5356de8e-63e0-4e06-afa3-9ce50dab54e6 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.261766 Sep 30 09:46:50.548999 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 62/130] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:50 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:50.596106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:50.605328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5210ef3e-df41-4438-a527-e0e703dcba29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.345927 Sep 30 09:46:50.606277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 69/131] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:50 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:51.311474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:51.311474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:51.317484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:51.317985 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:51.318440 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:51.319138 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:51.319574 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:51.319996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:51.337413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-0e738d0f-47c3-4b0e-8d9f-2788b533bb87 req-1c4b13e7-03d2-48f6-8b39-d295730a72e2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0bc1e701-2247-4837-81c0-289806279c31", "name": "network-vif-plugged", "status": "completed", "tag": "cc523046-219f-42b7-a7b9-700936140c81"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:51.349698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0e738d0f-47c3-4b0e-8d9f-2788b533bb87 req-1c4b13e7-03d2-48f6-8b39-d295730a72e2 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:51.350474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0e738d0f-47c3-4b0e-8d9f-2788b533bb87 req-1c4b13e7-03d2-48f6-8b39-d295730a72e2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:51.351129 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0e738d0f-47c3-4b0e-8d9f-2788b533bb87 req-1c4b13e7-03d2-48f6-8b39-d295730a72e2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:51.362739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-0e738d0f-47c3-4b0e-8d9f-2788b533bb87 req-1c4b13e7-03d2-48f6-8b39-d295730a72e2 service nova] Creating event network-vif-plugged:cc523046-219f-42b7-a7b9-700936140c81 for instance 0bc1e701-2247-4837-81c0-289806279c31 on hv-cinder-maste Sep 30 09:46:51.368908 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-0e738d0f-47c3-4b0e-8d9f-2788b533bb87 req-1c4b13e7-03d2-48f6-8b39-d295730a72e2 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035193 Sep 30 09:46:51.370026 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 70/132] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:46:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:51.471514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:51.483752 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-27d68360-070f-4aaa-9a03-042b70fea921 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.173594 Sep 30 09:46:51.483752 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 63/133] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:51 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:51.571497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:51.572250 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:51.593141 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:51.593632 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:51.594496 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:51.597365 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:51.598043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:51.598655 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:51.631120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:51.632206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:51.648228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:51.648228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:51.648228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:51.648498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:51.649450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:51.651671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:52.068819 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:52.077016 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a0961895-5007-4c68-b608-42f3ef57db5a tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1314 microversion: 2.1 time: 0.509734 Sep 30 09:46:52.077246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:52.077592 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 71/134] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:51 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1314 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:52.085409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-822e3a0b-6b51-4618-88b3-431b0f91ef2d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.459605 Sep 30 09:46:52.088041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 64/135] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:51 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:52.509317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:52.509317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:52.512330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:52.513024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:52.513460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:52.514288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:52.514767 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:52.515455 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:52.666964 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:52.674755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-51ce4181-466a-4665-932f-3d785e8af740 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.173399 Sep 30 09:46:52.675330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 72/136] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:52 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:53.101342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:53.102722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:53.112741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:53.114042 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:53.116169 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=81 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:53.126390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:53.129754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:53.129754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:53.129754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:53.129754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:53.130090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:53.133366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:53.134359 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:53.134359 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:53.135099 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:53.135533 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:53.135945 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:53.363376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:53.369781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:53.376272 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7f39aacf-e07b-447a-b213-c09acd82ce76 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1502 microversion: 2.1 time: 0.277517 Sep 30 09:46:53.379667 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 65/137] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:53 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1502 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:53.381683 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-44eaf9cf-b291-4b0c-9b0e-81f714a8dc39 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.273324 Sep 30 09:46:53.382734 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 73/138] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:53 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:53.698785 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:53.702776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:53.709830 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:53.710484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:53.710965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:53.712485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:53.712943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:53.713363 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:53.874502 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:53.882216 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9a4d61d1-db92-4248-a2fd-0ee1ed813fdc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.187311 Sep 30 09:46:53.882813 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 66/139] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:53 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:54.405797 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:54.406412 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:54.413955 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:54.414437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:54.414902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:54.418613 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:54.418613 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:54.418613 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:54.563579 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:54.571987 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bd52715f-71b9-4654-93e0-c8f85b1a924e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.170721 Sep 30 09:46:54.574661 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 74/140] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:54 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:54.905100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:54.906307 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:54.912776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:54.913268 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:54.913695 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:54.914408 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:54.914876 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:54.915285 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:55.106730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:55.106730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-25d370eb-e0c3-4600-be56-7c5468bb7b12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.188921 Sep 30 09:46:55.106730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 67/141] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:54 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:55.596449 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:55.597053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:55.602803 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:55.603284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:55.603709 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:55.604387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:55.604827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:55.605240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:55.769092 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:55.778176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dcf2bbc9-97d2-4e6e-a8c6-8b09b5070978 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.185956 Sep 30 09:46:55.778820 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 75/142] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:55 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:56.118261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:56.118924 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:56.125443 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:56.126133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:56.126568 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:56.127277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:56.127725 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:56.128144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:56.258504 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:56.266903 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-01b80fca-f0d3-4c96-811a-e03a8c9db8a2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.152439 Sep 30 09:46:56.267478 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 68/143] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:56 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:56.806148 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:56.807495 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:56.815819 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:56.816893 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:56.817765 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:56.819440 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:56.824525 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:56.824525 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:56.977730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:56.986252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-17385280-40e1-45eb-bb40-fbe033c77c4f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1368 microversion: 2.1 time: 0.186924 Sep 30 09:46:56.986847 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 76/144] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:56 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1368 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:57.289152 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:57.289748 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:57.295535 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:57.296008 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:57.296429 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:57.297100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:57.297544 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:57.297955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:57.444513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:57.452855 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1cba5548-5bf6-48ec-bef2-6e61b23749d8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.167464 Sep 30 09:46:57.453456 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 69/145] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:57 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:46:58.008186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:58.008782 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:58.015206 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:58.015484 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:58.015915 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:58.016717 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:58.017170 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:58.017587 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:58.158323 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:58.166342 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-241c89ba-b9cc-41ed-8c08-fac6bc5e1ed5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1558 microversion: 2.1 time: 0.161990 Sep 30 09:46:58.166939 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 77/146] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:58 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:58.191095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:58.191890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:58.202261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:58.203407 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:58.205045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:58.206393 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:58.206887 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:58.207308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:58.344846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-57a22033-c67e-4f57-b225-72b85677d0ac tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.153430 Sep 30 09:46:58.344846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 70/147] 10.222.0.39 () {58 vars in 1275 bytes} [Fri Sep 30 09:46:58 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-interface => generated 284 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:46:58.476983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:58.477651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:58.486407 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:58.488056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:58.489806 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:58.490730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:58.491344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:58.492187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:58.617393 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b3551656-cf4b-4d3b-a96e-f0599026be3e"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:46:58.620485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:58.628886 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:58.632066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:58.632066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:58.632066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:58.632066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:58.632776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:58.710114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:58.721912 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-923a69a3-9565-4ba3-8de6-01184e41440e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.249369 Sep 30 09:46:58.722731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 78/148] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:58 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:46:59.742394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-632d1284-e004-4bdc-a968-2418111b2836 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "POST /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.132403 Sep 30 09:46:59.744783 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 71/149] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:46:58 2022] POST /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41/os-volume_attachments => generated 194 bytes in 1137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:46:59.756524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:46:59.759994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:46:59.773060 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:59.773539 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:59.773969 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:59.776520 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:46:59.777318 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:46:59.777745 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:46:59.970741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:46:59.979715 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dc0f857b-155a-45f8-a05b-211a6c4fe31d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.238418 Sep 30 09:46:59.981669 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 79/150] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:46:59 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:00.160034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-25b4217e-a4e0-4189-855e-cec6f2a19e99 req-50719123-b4bb-4e45-92cb-baefb2d0de75 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0bc1e701-2247-4837-81c0-289806279c31", "tag": "cc523046-219f-42b7-a7b9-700936140c81"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:47:00.173980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-25b4217e-a4e0-4189-855e-cec6f2a19e99 req-50719123-b4bb-4e45-92cb-baefb2d0de75 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:00.176262 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-25b4217e-a4e0-4189-855e-cec6f2a19e99 req-50719123-b4bb-4e45-92cb-baefb2d0de75 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:00.176891 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-25b4217e-a4e0-4189-855e-cec6f2a19e99 req-50719123-b4bb-4e45-92cb-baefb2d0de75 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:00.189863 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-25b4217e-a4e0-4189-855e-cec6f2a19e99 req-50719123-b4bb-4e45-92cb-baefb2d0de75 service nova] Creating event network-changed:cc523046-219f-42b7-a7b9-700936140c81 for instance 0bc1e701-2247-4837-81c0-289806279c31 on hv-cinder-maste Sep 30 09:47:00.200853 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-25b4217e-a4e0-4189-855e-cec6f2a19e99 req-50719123-b4bb-4e45-92cb-baefb2d0de75 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042862 Sep 30 09:47:00.201539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 72/151] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:47:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:47:01.012329 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:01.013737 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:01.023231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:01.024365 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:01.025497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:01.026905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:01.028894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:01.030566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:01.208477 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:01.231837 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2fd6dfc1-35cc-4c60-bc85-ab8072a534dd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.223778 Sep 30 09:47:01.235123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 80/152] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:01 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:02.266411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:02.267961 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:02.275655 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=76 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:02.277699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:02.278663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:02.279928 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:02.281388 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:02.282274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:02.283242 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:02.434667 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:02.442483 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-65be2deb-2183-4d21-83c9-0f4afe27ae12 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.185994 Sep 30 09:47:02.443079 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 73/153] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:02 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:03.466354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:03.466992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:03.472767 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:03.473239 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:03.473661 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:03.474707 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:03.475156 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:03.475564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:03.611163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:03.615246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:03.622021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dad34d87-c9f6-46ae-b974-c6275867f854 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.159449 Sep 30 09:47:03.622628 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 81/154] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:03 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:04.645759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:04.646356 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:04.652793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:04.653270 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:04.653699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:04.654376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:04.654851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:04.655272 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:04.821515 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:04.831473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9bf742be-b2d6-4784-a063-810d0be7c0b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.189891 Sep 30 09:47:04.833710 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 74/155] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:04 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:05.856279 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:05.857300 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:05.864168 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:05.864652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:05.865083 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:05.865802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:05.866254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:05.866705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:06.001787 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:06.010931 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-14669606-87db-4344-8cb3-e13870f3cc6b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.157072 Sep 30 09:47:06.010931 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 82/156] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:05 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:07.038611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:07.043880 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:07.055149 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:07.055705 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:07.056372 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:07.057137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:07.057749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:07.058168 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:07.199664 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:07.207445 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f37d2457-bb94-4463-9c99-6b6cdec4b6a1 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.177895 Sep 30 09:47:07.214725 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 75/157] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:07 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:08.230135 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:08.230776 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:08.239212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:08.239212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:08.239212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:08.239889 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:08.240344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:08.240761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:08.376766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:08.386215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2cf2c069-2ece-4e6a-865a-b7d996d4f6ba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.159855 Sep 30 09:47:08.386810 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 83/158] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:08 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:09.411325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:09.418978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:09.419966 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:09.420444 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:09.421029 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:09.421718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:09.422166 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:09.422580 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:09.586169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:09.597292 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4a7210fd-df56-422d-8c9a-e4203b4d91d6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.189859 Sep 30 09:47:09.599496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 76/159] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:09 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:10.362940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:10.368168 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:10.370066 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:10.370766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:10.371198 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:10.371898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:10.372387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:10.372811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:10.398175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:47:10.621806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:10.622497 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:10.629933 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:10.630627 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:10.631060 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:10.631752 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:10.632201 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:10.632618 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:10.636771 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:10.693277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-cbc99c25-a380-4ad2-bce8-b140031f2912 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "DELETE /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 204 len: 0 microversion: 2.1 time: 0.334155 Sep 30 09:47:10.694158 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 84/160] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:47:10 2022] DELETE /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:47:10.703526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:10.705417 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:10.710842 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:10.711335 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:10.711773 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:10.712472 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:10.712925 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:10.713345 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:10.801786 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:10.809072 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-fd784e46-384d-469d-85c8-d32eee03161f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.191512 Sep 30 09:47:10.809692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 77/161] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:10 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:10.866195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:10.874836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-31392ba6-67ac-428f-a268-9d7b77994f26 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1554 microversion: 2.1 time: 0.176260 Sep 30 09:47:10.875405 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 85/162] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:10 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1554 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:11.833106 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:11.834367 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:11.842462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:11.843270 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:11.843759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:11.844576 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:11.845091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:11.845572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:11.898871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:11.900238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:11.915955 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:11.916446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:11.916868 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:11.917735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:11.918176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:11.918639 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:12.066451 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:12.078194 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-57433c8f-cd8c-4e5c-bf04-e9666c7f35ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.249328 Sep 30 09:47:12.078989 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 78/163] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:11 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:12.096913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:12.105286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8ce2b700-a907-4439-9560-ea32f9746a49 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1554 microversion: 2.1 time: 0.210359 Sep 30 09:47:12.105861 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 86/164] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:11 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1554 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:13.101278 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:13.102131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:13.108869 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:13.109664 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:13.110197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:13.111041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:13.111505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:13.111927 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:13.139259 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:13.139863 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:13.146188 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:13.160372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:13.160372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:13.160372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:13.160372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:13.161251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:13.504215 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:13.513590 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d38e06cf-9deb-475e-ac02-e2571f720202 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.416131 Sep 30 09:47:13.515972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 79/165] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:13 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:13.520304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:13.533374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3b0eb025-bb23-4d12-8ed9-3db618b87499 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1554 microversion: 2.1 time: 0.400382 Sep 30 09:47:13.534293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 87/166] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:13 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1554 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:13.974404 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-db8de443-2e9e-4ce0-8b3f-a281cf98bcf5 req-572ccafa-2d4b-41f1-9322-7caecd417120 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2a244aaa-c28f-405e-ad45-39386205bb41", "name": "network-vif-deleted", "tag": "8588e984-7119-40f8-9e2a-800c04c4c6aa"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:47:13.985364 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-db8de443-2e9e-4ce0-8b3f-a281cf98bcf5 req-572ccafa-2d4b-41f1-9322-7caecd417120 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:13.985844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-db8de443-2e9e-4ce0-8b3f-a281cf98bcf5 req-572ccafa-2d4b-41f1-9322-7caecd417120 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:13.986269 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-db8de443-2e9e-4ce0-8b3f-a281cf98bcf5 req-572ccafa-2d4b-41f1-9322-7caecd417120 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:13.998467 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-db8de443-2e9e-4ce0-8b3f-a281cf98bcf5 req-572ccafa-2d4b-41f1-9322-7caecd417120 service nova] Creating event network-vif-deleted:8588e984-7119-40f8-9e2a-800c04c4c6aa for instance 2a244aaa-c28f-405e-ad45-39386205bb41 on hv-cinder-maste Sep 30 09:47:14.003708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-db8de443-2e9e-4ce0-8b3f-a281cf98bcf5 req-572ccafa-2d4b-41f1-9322-7caecd417120 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033047 Sep 30 09:47:14.004420 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 80/167] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:47:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:14.537243 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:14.537849 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:14.543728 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:14.544207 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:14.544640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:14.545322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:14.545778 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:14.546199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:14.558056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:14.559012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:14.565125 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:14.565600 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:14.566027 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:14.566743 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:14.567195 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:14.568043 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:14.665120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:14.682480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-59160f23-f253-4343-9da2-04d3c9f65e58 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1345 microversion: 2.1 time: 0.128689 Sep 30 09:47:14.683645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 81/168] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:14 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1345 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:14.756330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:14.764623 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5be29ad4-757b-4708-8d74-4f4777128c85 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.230780 Sep 30 09:47:14.765199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 88/169] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:14 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:15.705249 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:15.713115 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:15.713936 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:15.714482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:15.715163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:15.716039 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:15.716672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:15.717870 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:15.787497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:15.788747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:15.792309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:15.795271 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:15.798173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:15.799890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lazy-loading 'fault' on Instance uuid 2a244aaa-c28f-405e-ad45-39386205bb41 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:47:15.803425 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:15.804106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:15.805000 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:15.806491 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:15.807183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:15.807620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:15.912916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ae7915e1b2474d9384a0fdcb2ea177b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b7f6b7ca2f3b4c20ab58940f7e24c1ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:15.923056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7101211f-bff3-4ccc-a596-20f089ef843b tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 200 len: 1350 microversion: 2.1 time: 0.221538 Sep 30 09:47:15.923654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 82/170] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:15 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 1350 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:16.232896 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:16.240773 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-28b9ccb1-c886-4b72-9a4b-a72ecee8ec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.457653 Sep 30 09:47:16.245582 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 89/171] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:15 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:16.957947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:16.959439 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:17.002007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:17.006473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.034s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:17.007705 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:17.009816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:17.010466 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:17.010921 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:17.028150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] HTTP exception thrown: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Request method failure captured: Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 HTTP/1.1 Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrXpQmiNmVNMGFbFcwFsxCv1xAyuAUshgwId7GPh_OKrQ2KrmVYgLfH-xRbSuEu4XCieZcI1Dg2V8IxN7MyKTU8G8iMEvU5CPFkovCaHENMtHVnM6i261DluqCTxT1wyGfJxjrr9nT8LyFDktaeD7gYspCPMRKDNvblL0Pu3uabEI8 Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: b7f6b7ca2f3b4c20ab58940f7e24c1ed Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:47:17.033699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: b7f6b7ca2f3b4c20ab58940f7e24c1ed Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-DeleteServersTestJSON-1102723588-project Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: ae7915e1b2474d9384a0fdcb2ea177b6 Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-DeleteServersTestJSON-1102723588-project Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.035276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '2a244aaa-c28f-405e-ad45-39386205bb41'} Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:17.036137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.037461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.039895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Returning 404 to user: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:17.040766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4ac033f5-9663-4795-8494-938a4d065002 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 404 len: 111 microversion: 2.1 time: 0.092946 Sep 30 09:47:17.042940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 83/172] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:16 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 111 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:47:17.246651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:17.249447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:17.266312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:17.267939 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:17.268601 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:17.269458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:17.269830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:17.270304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:17.270498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:17.271789 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:17.288925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:17.289663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:17.290254 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:17.291522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:17.292608 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:17.293184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:17.293483 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] HTTP exception thrown: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Request method failure captured: Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41/os-volume_attachments HTTP/1.1 Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrXpQmiNmVNMGFbFcwFsxCv1xAyuAUshgwId7GPh_OKrQ2KrmVYgLfH-xRbSuEu4XCieZcI1Dg2V8IxN7MyKTU8G8iMEvU5CPFkovCaHENMtHVnM6i261DluqCTxT1wyGfJxjrr9nT8LyFDktaeD7gYspCPMRKDNvblL0Pu3uabEI8 Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: b7f6b7ca2f3b4c20ab58940f7e24c1ed Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:47:17.300446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: b7f6b7ca2f3b4c20ab58940f7e24c1ed Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-DeleteServersTestJSON-1102723588-project Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: ae7915e1b2474d9384a0fdcb2ea177b6 Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-DeleteServersTestJSON-1102723588-project Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.302102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'server_id': '2a244aaa-c28f-405e-ad45-39386205bb41'} Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:17.303173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:17.304572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:17.306353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Returning 404 to user: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:17.306353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-223cf0d6-df3b-4538-9df5-302b3d2240f3 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.065171 Sep 30 09:47:17.306353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 90/173] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:47:17 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41/os-volume_attachments => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:47:17.542705 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:17.548714 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ffb63772-83b8-4deb-9ba4-2fc9a485c7b4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.285837 Sep 30 09:47:17.549393 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 84/174] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:17 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:18.572360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:18.572956 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:18.580547 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:18.581029 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:18.581446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:18.582163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:18.582900 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:18.584448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:18.726912 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:18.735889 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2da6a5ca-4139-4e3b-a645-b7a7e3c6ab6a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.167391 Sep 30 09:47:18.736462 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 91/175] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:18 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:18.967184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:18.967673 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:18.973757 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:18.974407 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:18.974874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:18.975872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:18.976565 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:18.977131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:18.995889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] HTTP exception thrown: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Request method failure captured: Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: DELETE /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 HTTP/1.1 Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrXpQmiNmVNMGFbFcwFsxCv1xAyuAUshgwId7GPh_OKrQ2KrmVYgLfH-xRbSuEu4XCieZcI1Dg2V8IxN7MyKTU8G8iMEvU5CPFkovCaHENMtHVnM6i261DluqCTxT1wyGfJxjrr9nT8LyFDktaeD7gYspCPMRKDNvblL0Pu3uabEI8 Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: b7f6b7ca2f3b4c20ab58940f7e24c1ed Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:47:18.998931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: b7f6b7ca2f3b4c20ab58940f7e24c1ed Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-DeleteServersTestJSON-1102723588-project Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: ae7915e1b2474d9384a0fdcb2ea177b6 Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-DeleteServersTestJSON-1102723588-project Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.000791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '2a244aaa-c28f-405e-ad45-39386205bb41'} Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:19.001681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.003192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.005024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Returning 404 to user: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:19.005024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8421b0b2-fbdf-491b-a233-86737fe44d52 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "DELETE /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 404 len: 111 microversion: 2.1 time: 0.038261 Sep 30 09:47:19.005024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 85/176] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:47:18 2022] DELETE /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:47:19.010957 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:19.011645 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] [instance: 2a244aaa-c28f-405e-ad45-39386205bb41] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:19.020364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:19.021149 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:19.021584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:19.022496 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:19.023048 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:19.023605 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:19.038554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] HTTP exception thrown: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Request method failure captured: Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 HTTP/1.1 Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrXpQmiNmVNMGFbFcwFsxCv1xAyuAUshgwId7GPh_OKrQ2KrmVYgLfH-xRbSuEu4XCieZcI1Dg2V8IxN7MyKTU8G8iMEvU5CPFkovCaHENMtHVnM6i261DluqCTxT1wyGfJxjrr9nT8LyFDktaeD7gYspCPMRKDNvblL0Pu3uabEI8 Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: b7f6b7ca2f3b4c20ab58940f7e24c1ed Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:47:19.041057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b7f6b7ca2f3b4c20ab58940f7e24c1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: b7f6b7ca2f3b4c20ab58940f7e24c1ed Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-DeleteServersTestJSON-1102723588 Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-DeleteServersTestJSON-1102723588-project Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: ae7915e1b2474d9384a0fdcb2ea177b6 Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-DeleteServersTestJSON-1102723588-project Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.042390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '2a244aaa-c28f-405e-ad45-39386205bb41'} Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:19.043254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. Sep 30 09:47:19.044701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:19.047407 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] Returning 404 to user: Instance 2a244aaa-c28f-405e-ad45-39386205bb41 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:19.048310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-53b6e216-145c-4b64-8dad-44d61202bfb5 tempest-DeleteServersTestJSON-1102723588 tempest-DeleteServersTestJSON-1102723588-project] 10.222.0.39 "GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41" status: 404 len: 111 microversion: 2.1 time: 0.042342 Sep 30 09:47:19.049198 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 92/177] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:19 2022] GET /compute/v2.1/servers/2a244aaa-c28f-405e-ad45-39386205bb41 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:47:19.760639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:19.766120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:19.768759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:19.769523 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:19.769952 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:19.770670 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:19.773184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:19.773633 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:20.059547 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:20.077416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-baa9ba16-2ab2-4d34-a9c3-d8ee920131a5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.319713 Sep 30 09:47:20.081990 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 86/178] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:19 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:21.108019 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:21.111568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:21.126136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:21.128331 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:21.128967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:21.129850 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:21.130459 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:21.131082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:21.354333 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:21.364827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d1ef0694-8d2e-4577-aa51-14a829598b73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.268341 Sep 30 09:47:21.366032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 93/179] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:21 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:22.389963 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:22.390537 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:22.396886 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:22.397405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:22.397876 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:22.398683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:22.399370 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:22.400004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:22.614046 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:22.622546 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-79e7bf91-e039-4ca7-9811-01ac0b994622 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.237121 Sep 30 09:47:22.623846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 87/180] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:22 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:23.655505 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:23.657906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:23.676466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:23.680038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:23.680038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:23.680038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:23.680038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:23.680444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:23.933537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:23.944227 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c0e66232-3bd4-480e-aa3f-179c52ed2b37 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.294337 Sep 30 09:47:23.945626 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 94/181] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:23 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:24.969461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:24.970066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:24.976847 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:24.977791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:24.978227 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:24.978982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:24.979441 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:24.979867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:25.115440 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:25.124113 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ff053c09-466e-4c0c-8dce-537b8d180676 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.159096 Sep 30 09:47:25.124681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 88/182] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:24 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:26.147140 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:26.148453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:26.150836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:26.156412 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:26.157184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:26.157654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:26.158350 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:26.158829 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:26.159253 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:26.300677 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:26.308264 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1d3eaf4e-d489-488c-862d-08a5ef0c016c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.165733 Sep 30 09:47:26.308826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 95/183] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:26 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:27.331475 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:27.341094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:27.341094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:27.341094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:27.341094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:27.342437 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:27.342437 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:27.342437 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:27.572244 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:27.583711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-380eb65c-d3da-4162-9c3a-0141d6dd8ca9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.256082 Sep 30 09:47:27.587842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 89/184] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:27 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:27.974471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fff847dd-86bc-4738-b347-18a0271eb0ae tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Action: 'create', calling method: .version_select at 0x7f72c0e47d30>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1816023959"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:47:28.611155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:28.611750 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:28.620962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:28.620962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:28.620962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:28.620962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:28.621308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:28.621308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:28.748145 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:28.757844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5b7f23a8-f785-47fb-83ff-683b85290ee4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.149051 Sep 30 09:47:28.757844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 90/185] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:28 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:28.964699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fff847dd-86bc-4738-b347-18a0271eb0ae tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 1.237375 Sep 30 09:47:28.965391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 96/186] 10.222.0.39 () {60 vars in 1155 bytes} [Fri Sep 30 09:47:27 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 1239 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 30 09:47:28.978283 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a401f591-6014-49d2-b7a4-cdd9d59dca99 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d135e0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:29.029374 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a401f591-6014-49d2-b7a4-cdd9d59dca99 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 200 len: 345 microversion: 2.1 time: 0.054016 Sep 30 09:47:29.033643 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 91/187] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:28 2022] GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 345 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:29.778614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:29.779875 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:29.785219 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:29.785712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:29.786151 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:29.786880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:29.787337 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:29.787765 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:29.955591 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:29.963760 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c6c96c9e-53a6-46ab-91b7-fd2ebbb22565 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.188902 Sep 30 09:47:29.964318 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 97/188] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:29 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:30.044625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-153c8066-e773-41ee-a083-5fa708a3d579 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d78310>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:30.104818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-153c8066-e773-41ee-a083-5fa708a3d579 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 200 len: 345 microversion: 2.1 time: 0.064962 Sep 30 09:47:30.105387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 92/189] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:30 2022] GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 345 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:30.988401 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:30.988942 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:30.995141 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:30.995638 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:30.996075 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:30.996771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:30.997805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:30.998304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:31.120758 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2e1e8116-a915-4937-b523-d5c270958eeb tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d76310>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:31.132369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:31.143213 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3c786002-43fa-4e05-83ee-ff4fe53129d3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.158779 Sep 30 09:47:31.144021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 98/190] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:30 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:31.180137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2e1e8116-a915-4937-b523-d5c270958eeb tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 200 len: 345 microversion: 2.1 time: 0.064247 Sep 30 09:47:31.181153 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 93/191] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:31 2022] GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 345 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:32.168623 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:32.169847 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:32.175805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:32.176300 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:32.176719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:32.177586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:32.178041 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:32.178464 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:32.182661 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:32.200178 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-545b6b27-757c-4d11-9818-006035469f67 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d78790>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:32.254209 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-545b6b27-757c-4d11-9818-006035469f67 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 200 len: 345 microversion: 2.1 time: 0.062441 Sep 30 09:47:32.261722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 94/192] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:32 2022] GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 345 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:47:32.316471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:32.324050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e12357e7-5cbf-471a-a988-3e260451393a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1522 microversion: 2.1 time: 0.159361 Sep 30 09:47:32.324614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 99/193] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:32 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:33.268527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-03107918-69dd-47de-bf11-e20c1bb4b06b tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d74c10>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:33.348346 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:33.348971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:33.355124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:33.355602 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:33.356030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:33.357201 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:33.357649 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:33.358234 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:33.491210 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:33.498972 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4a5fb804-1af9-481e-853c-4fe76a64999d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1502 microversion: 2.1 time: 0.154754 Sep 30 09:47:33.499533 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 100/194] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:33 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1502 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:33.524676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:33.525194 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:33.531443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:33.531920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:33.532441 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:33.533139 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:33.533577 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:33.533993 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:33.660007 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:33.667999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9789f149-e82e-45cc-b8f8-77a6bf785f26 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1515 microversion: 2.1 time: 0.147058 Sep 30 09:47:33.668555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 101/195] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:33 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1515 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:33.738969 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-03107918-69dd-47de-bf11-e20c1bb4b06b tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 200 len: 346 microversion: 2.1 time: 0.465610 Sep 30 09:47:33.738969 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 95/196] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:33 2022] GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 346 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:33.745054 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-06c892e4-495f-4c58-b42e-98bc4b815f45 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Action: 'create', calling method: .version_select at 0x7f72c0e47d30>, body: {"snapshot": {"volume_id": "834c8556-44d8-4b4f-a9c7-7c1d6a94136e", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-639741644"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:47:34.298742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-06c892e4-495f-4c58-b42e-98bc4b815f45 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.558710 Sep 30 09:47:34.299357 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 102/197] 10.222.0.39 () {60 vars in 1162 bytes} [Fri Sep 30 09:47:33 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:34.313584 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2043e85a-2473-4573-bd53-58f4c322eac0 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d76f70>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:34.364308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2043e85a-2473-4573-bd53-58f4c322eac0 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.056206 Sep 30 09:47:34.365124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 96/198] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:34 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:34.378028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4860cd34-d5ba-47f7-9c9e-cc49629d822d tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d46790>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:34.413945 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4860cd34-d5ba-47f7-9c9e-cc49629d822d tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.040708 Sep 30 09:47:34.414716 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 103/199] 10.222.0.39 () {58 vars in 1140 bytes} [Fri Sep 30 09:47:34 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:34.425590 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0c9117a6-d025-4b19-bc12-a309a79f7673 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d76e50>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:34.474520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0c9117a6-d025-4b19-bc12-a309a79f7673 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.052600 Sep 30 09:47:34.475131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 97/200] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:34 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:35.487694 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-693af2bb-d708-4131-af7a-c96430279a4b tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0decee0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:35.520022 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-693af2bb-d708-4131-af7a-c96430279a4b tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.036097 Sep 30 09:47:35.520585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 104/201] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:35 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:36.533249 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f9a90e76-86ab-4ca1-9d4d-03fdb563d4c6 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d78310>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:36.563526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f9a90e76-86ab-4ca1-9d4d-03fdb563d4c6 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.034631 Sep 30 09:47:36.564284 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 98/202] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:36 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:37.585431 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b3232921-34e9-497b-8284-d58b2c86f218 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0dec940>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:37.661318 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b3232921-34e9-497b-8284-d58b2c86f218 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.079882 Sep 30 09:47:37.661883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 105/203] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:37 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:38.670985 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=37 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:38.675236 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-15108896-1f30-4d46-b5f1-5fba95bf90e8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d13790>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:38.705173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-15108896-1f30-4d46-b5f1-5fba95bf90e8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.035840 Sep 30 09:47:38.705741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 99/204] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:38 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:39.717953 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e3ea0957-7c60-46c7-a645-f7f32361ee4e tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0dec1f0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:39.747447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e3ea0957-7c60-46c7-a645-f7f32361ee4e tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.033905 Sep 30 09:47:39.748556 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 106/205] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:39 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:40.756368 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=72 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:40.760913 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3cc579e9-7bd6-44f9-8f8b-f401c2ca0144 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d78670>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:40.793264 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3cc579e9-7bd6-44f9-8f8b-f401c2ca0144 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.037502 Sep 30 09:47:40.793977 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 100/206] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:40 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:41.805456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-50e52d7c-8bc4-4161-af58-2f5c0feb3571 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0e8db80>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:41.842242 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-50e52d7c-8bc4-4161-af58-2f5c0feb3571 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.040697 Sep 30 09:47:41.843665 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 107/207] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:41 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:42.857483 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d9a5755d-5d28-425d-b68d-7dd3624bd9f8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d76e50>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:42.888335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d9a5755d-5d28-425d-b68d-7dd3624bd9f8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.034967 Sep 30 09:47:42.888940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 101/208] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:42 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:47:43.898150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=11 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:43.902118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9bb0bb4f-fe0b-456e-a5fc-9da687643fb9 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0e47b80>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:43.933948 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9bb0bb4f-fe0b-456e-a5fc-9da687643fb9 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 287 microversion: 2.1 time: 0.036813 Sep 30 09:47:43.934704 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 108/209] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:43 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:43.946602 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-40ddecb8-5907-4dcd-8f3e-f0887a06d643 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0e91550>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:44.080326 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-40ddecb8-5907-4dcd-8f3e-f0887a06d643 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "DELETE /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 202 len: 0 microversion: 2.1 time: 0.138394 Sep 30 09:47:44.080896 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 102/210] 10.222.0.39 () {58 vars in 1254 bytes} [Fri Sep 30 09:47:43 2022] DELETE /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 0 bytes in 140 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:47:44.088982 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3ce16ae9-a393-4059-91d2-eca93d775a4c tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0e47dc0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:44.119907 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3ce16ae9-a393-4059-91d2-eca93d775a4c tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.034584 Sep 30 09:47:44.120491 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 109/211] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:44 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:45.133062 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-61c2f0f5-6d37-4fca-a13a-67298e58a5c1 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d78040>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:45.161815 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-61c2f0f5-6d37-4fca-a13a-67298e58a5c1 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.033000 Sep 30 09:47:45.162376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 103/212] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:45 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:46.171552 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=16 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:46.175569 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-97a507f8-942a-4f97-99a4-a22344580276 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0decca0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:46.206423 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-97a507f8-942a-4f97-99a4-a22344580276 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.035494 Sep 30 09:47:46.207231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 110/213] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:46 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:47.178607 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:47:47.180864 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:47.187664 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:47.188112 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:47.188540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:47.189230 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:47.189678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:47.190206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:47.220016 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c354abb5-8f1a-4f12-a30f-1491eb9e5c71 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0dec700>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:47.227597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lazy-loading 'flavor' on Instance uuid fd8e4eb8-1261-4abf-82ec-e1d8afda21ec {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:47:47.254976 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c354abb5-8f1a-4f12-a30f-1491eb9e5c71 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 200 len: 286 microversion: 2.1 time: 0.039516 Sep 30 09:47:47.258913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 111/214] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:47 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 286 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:47.719868 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e8c5f61a-aafa-4dfc-bf6a-c2d9fce804ab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec/action" status: 200 len: 32 microversion: 2.1 time: 0.544261 Sep 30 09:47:47.719868 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 104/215] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:47:47 2022] POST /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec/action => generated 32 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 09:47:47.728371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:47.729055 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:47.735904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:47.736402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:47.737017 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:47.737733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:47.738383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:47.738839 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:47.884189 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:47.892431 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0ebffed1-cded-439a-850b-89ef63bdb09e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.168312 Sep 30 09:47:47.893006 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 112/216] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:47 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:48.268742 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2db9202e-4aa7-4783-96fa-eb62dee1a13f tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d13ca0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:48.300540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-2db9202e-4aa7-4783-96fa-eb62dee1a13f tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] HTTP exception thrown: Snapshot 4a938800-46b4-4ed1-a25d-68f952aee227 could not be found. Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2db9202e-4aa7-4783-96fa-eb62dee1a13f tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Request method failure captured: Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 HTTP/1.1 Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrsvBfs5XdC4gRgZFsavn_LgKC2H3nv-9glsDMvGoUIDArg3nhXh3_AVX8O2gr6ijdk7rktjgvoVXW4CnvdVO9gDJIrW0lfWg8WLVpqBANqfCpbyGG4YhssSEvVY0xc4kBt9wftkvb9UdcAjf3PdQnsIGM37aDhu8AaDVMOrNsK-oq0 Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 0860efc0d2584be78c2b46f937ebe7cf Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:47:48.313958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 0860efc0d2584be78c2b46f937ebe7cf Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-VolumesSnapshotsTestJSON-1101521384-project Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 6f80ef9b0f0440eeade62a9b6070db35 Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1101521384-project Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Snapshot 4a938800-46b4-4ed1-a25d-68f952aee227 could not be found. Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '4a938800-46b4-4ed1-a25d-68f952aee227'} Sep 30 09:47:48.315524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 4a938800-46b4-4ed1-a25d-68f952aee227 could not be found. (HTTP 404) (Request-ID: req-71e4051c-35f2-4c95-a788-82e59dea02b4) Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:48.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 09:47:48.317621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 4a938800-46b4-4ed1-a25d-68f952aee227 could not be found. Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 30 09:47:48.318685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 4a938800-46b4-4ed1-a25d-68f952aee227 could not be found. Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 4a938800-46b4-4ed1-a25d-68f952aee227 could not be found. Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2db9202e-4aa7-4783-96fa-eb62dee1a13f tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Returning 404 to user: Snapshot 4a938800-46b4-4ed1-a25d-68f952aee227 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:48.319741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2db9202e-4aa7-4783-96fa-eb62dee1a13f tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227" status: 404 len: 111 microversion: 2.1 time: 0.052184 Sep 30 09:47:48.320447 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 105/217] 10.222.0.39 () {58 vars in 1251 bytes} [Fri Sep 30 09:47:48 2022] GET /compute/v2.1/os-snapshots/4a938800-46b4-4ed1-a25d-68f952aee227 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:47:48.330140 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-623143d5-c57c-41d9-8250-b71186f3b661 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0e8db80>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:48.593637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-623143d5-c57c-41d9-8250-b71186f3b661 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 202 len: 0 microversion: 2.1 time: 0.269766 Sep 30 09:47:48.594292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 113/218] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 09:47:48 2022] DELETE /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 0 bytes in 271 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:47:48.602687 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9c6a7dab-b24b-44ba-aa61-fcafd1a40832 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0e99d30>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:48.772698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9c6a7dab-b24b-44ba-aa61-fcafd1a40832 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 200 len: 345 microversion: 2.1 time: 0.165436 Sep 30 09:47:48.772698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 106/219] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:48 2022] GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 345 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:47:48.915627 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:48.916236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:48.922092 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:48.922570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:48.923033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:48.923719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:48.924174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:48.924612 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:49.058756 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:49.066562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2bbf3bb8-b987-412c-ab26-072ec13fa018 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.154701 Sep 30 09:47:49.067171 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 114/220] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:48 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:49.780478 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-76fe3805-4dca-449c-a4bf-60f8ff8c8ca8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d769d0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:49.809991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-76fe3805-4dca-449c-a4bf-60f8ff8c8ca8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] HTTP exception thrown: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-76fe3805-4dca-449c-a4bf-60f8ff8c8ca8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Request method failure captured: Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e HTTP/1.1 Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrsvBfs5XdC4gRgZFsavn_LgKC2H3nv-9glsDMvGoUIDArg3nhXh3_AVX8O2gr6ijdk7rktjgvoVXW4CnvdVO9gDJIrW0lfWg8WLVpqBANqfCpbyGG4YhssSEvVY0xc4kBt9wftkvb9UdcAjf3PdQnsIGM37aDhu8AaDVMOrNsK-oq0 Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 0860efc0d2584be78c2b46f937ebe7cf Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:47:49.813091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 0860efc0d2584be78c2b46f937ebe7cf Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-VolumesSnapshotsTestJSON-1101521384-project Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 6f80ef9b0f0440eeade62a9b6070db35 Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1101521384-project Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '834c8556-44d8-4b4f-a9c7-7c1d6a94136e'} Sep 30 09:47:49.814696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. (HTTP 404) (Request-ID: req-1803dc2a-8405-4527-9c08-cd51f9d402fc) Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.815644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 09:47:49.817011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 30 09:47:49.818449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-76fe3805-4dca-449c-a4bf-60f8ff8c8ca8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Returning 404 to user: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-76fe3805-4dca-449c-a4bf-60f8ff8c8ca8 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 404 len: 109 microversion: 2.1 time: 0.038655 Sep 30 09:47:49.819700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 107/221] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:49 2022] GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:47:49.827218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ff79428c-47d8-4b9d-aac7-6c501ead84cd tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0e4b160>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:49.855920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-ff79428c-47d8-4b9d-aac7-6c501ead84cd tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] HTTP exception thrown: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ff79428c-47d8-4b9d-aac7-6c501ead84cd tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Request method failure captured: Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e HTTP/1.1 Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrsvBfs5XdC4gRgZFsavn_LgKC2H3nv-9glsDMvGoUIDArg3nhXh3_AVX8O2gr6ijdk7rktjgvoVXW4CnvdVO9gDJIrW0lfWg8WLVpqBANqfCpbyGG4YhssSEvVY0xc4kBt9wftkvb9UdcAjf3PdQnsIGM37aDhu8AaDVMOrNsK-oq0 Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 0860efc0d2584be78c2b46f937ebe7cf Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:47:49.860067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 0860efc0d2584be78c2b46f937ebe7cf Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-VolumesSnapshotsTestJSON-1101521384-project Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 6f80ef9b0f0440eeade62a9b6070db35 Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1101521384-project Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '834c8556-44d8-4b4f-a9c7-7c1d6a94136e'} Sep 30 09:47:49.861527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. (HTTP 404) (Request-ID: req-9ea23aeb-7892-437e-8695-8c58ab2e023c) Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.865296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 30 09:47:49.866667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 30 09:47:49.867910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ff79428c-47d8-4b9d-aac7-6c501ead84cd tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Returning 404 to user: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ff79428c-47d8-4b9d-aac7-6c501ead84cd tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 404 len: 109 microversion: 2.1 time: 0.038908 Sep 30 09:47:49.869419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 115/222] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 09:47:49 2022] DELETE /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:47:49.870363 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cfbfef45-9876-40ac-96ee-e27ab1ed1934 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Calling method '.version_select at 0x7f72c0d76a60>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:49.897639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-cfbfef45-9876-40ac-96ee-e27ab1ed1934 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] HTTP exception thrown: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cfbfef45-9876-40ac-96ee-e27ab1ed1934 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Request method failure captured: Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e HTTP/1.1 Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrsvBfs5XdC4gRgZFsavn_LgKC2H3nv-9glsDMvGoUIDArg3nhXh3_AVX8O2gr6ijdk7rktjgvoVXW4CnvdVO9gDJIrW0lfWg8WLVpqBANqfCpbyGG4YhssSEvVY0xc4kBt9wftkvb9UdcAjf3PdQnsIGM37aDhu8AaDVMOrNsK-oq0 Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 0860efc0d2584be78c2b46f937ebe7cf Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:47:49.899617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_0860efc0d2584be78c2b46f937ebe7cf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 0860efc0d2584be78c2b46f937ebe7cf Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1101521384 Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-VolumesSnapshotsTestJSON-1101521384-project Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 6f80ef9b0f0440eeade62a9b6070db35 Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1101521384-project Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '834c8556-44d8-4b4f-a9c7-7c1d6a94136e'} Sep 30 09:47:49.901035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. (HTTP 404) (Request-ID: req-501a68ee-85ac-4676-9b24-0c6d2946e837) Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.901914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 09:47:49.903214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 30 09:47:49.904480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cfbfef45-9876-40ac-96ee-e27ab1ed1934 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] Returning 404 to user: Volume 834c8556-44d8-4b4f-a9c7-7c1d6a94136e could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-cfbfef45-9876-40ac-96ee-e27ab1ed1934 tempest-VolumesSnapshotsTestJSON-1101521384 tempest-VolumesSnapshotsTestJSON-1101521384-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e" status: 404 len: 109 microversion: 2.1 time: 0.034982 Sep 30 09:47:49.906246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 108/223] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:49 2022] GET /compute/v2.1/os-volumes/834c8556-44d8-4b4f-a9c7-7c1d6a94136e => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 30 09:47:50.091240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:50.091862 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:50.099829 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:50.100652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:50.101306 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:50.102016 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:50.102482 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:50.102963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:50.328153 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:50.336336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a5708cdf-a1f1-4142-8d9b-88810404a5a4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.249484 Sep 30 09:47:50.337174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 116/224] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:50 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:51.366618 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:51.371261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:51.394656 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:51.399044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:51.399044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:51.401868 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:51.402452 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:51.406093 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:51.604206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:51.612435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4ef3aca8-178c-4957-897f-cad9a216858a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.251605 Sep 30 09:47:51.613138 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 109/225] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:51 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:52.635794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:52.637589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:52.649524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:52.649524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:52.649524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:52.649524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:52.649822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:52.649822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:52.792807 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:52.805127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5e821d23-15dc-4c7e-9f50-263df4acb772 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.172993 Sep 30 09:47:52.806186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 117/226] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:52 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:53.839327 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:53.841462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:53.855904 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:53.856955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:53.857390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:53.858107 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:53.859110 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:53.859681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:54.277612 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:54.299583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-230e2a7c-d8aa-461c-beb2-d6773f986211 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.470145 Sep 30 09:47:54.300266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 110/227] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:53 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:54.555768 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-eef29b96-aa4e-4006-8920-dffd3f4b3cbb tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Action: 'create', calling method: .version_select at 0x7f72c0cad040>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1901645948", "metadata": {"Type": "work"}}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:47:55.250283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-eef29b96-aa4e-4006-8920-dffd3f4b3cbb tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 1.142580 Sep 30 09:47:55.251093 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 118/228] 10.222.0.39 () {60 vars in 1156 bytes} [Fri Sep 30 09:47:54 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 1144 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Sep 30 09:47:55.271816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e33ce1d8-612e-4fb1-9b14-aa62980056c1 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Calling method '.version_select at 0x7f72c0cc1b80>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:55.325512 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:55.326087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:55.339574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:55.340289 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:55.340897 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:55.341752 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:55.342888 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:55.343303 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:55.521428 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e33ce1d8-612e-4fb1-9b14-aa62980056c1 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a" status: 200 len: 353 microversion: 2.1 time: 0.253581 Sep 30 09:47:55.521994 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 111/229] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:55 2022] GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a => generated 353 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:55.544755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:55.554299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f4fff575-f0e7-42d3-849d-64b7f39c1d2f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.232912 Sep 30 09:47:55.557190 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 119/230] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:55 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:56.541773 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-97c89bbe-8e1b-433f-909b-d1263b2463bf tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Calling method '.version_select at 0x7f72c0d7b310>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:56.579236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:56.580268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:56.587661 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:56.588180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:56.588626 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:56.594969 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-97c89bbe-8e1b-433f-909b-d1263b2463bf tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a" status: 200 len: 354 microversion: 2.1 time: 0.056862 Sep 30 09:47:56.595484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 112/231] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:56 2022] GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a => generated 354 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:47:56.597091 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:56.597614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:56.598057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:56.610209 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-52f882c8-6469-4577-a40a-c29660cc17de tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Calling method '.version_select at 0x7f72c0d7bca0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:56.688289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-52f882c8-6469-4577-a40a-c29660cc17de tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a" status: 200 len: 354 microversion: 2.1 time: 0.079990 Sep 30 09:47:56.688289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 113/232] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:56 2022] GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a => generated 354 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:47:56.707496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d0047002-e1d0-4207-8668-4566a952c1a2 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Calling method '.version_select at 0x7f72c0cc1a60>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:56.838747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:56.847402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0ad22a6e-0962-422a-96f6-3e721503478a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.273828 Sep 30 09:47:56.848295 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 120/233] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:56 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:47:56.921982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d0047002-e1d0-4207-8668-4566a952c1a2 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a" status: 202 len: 0 microversion: 2.1 time: 0.217148 Sep 30 09:47:56.921982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 114/234] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 09:47:56 2022] DELETE /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a => generated 0 bytes in 219 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 30 09:47:56.929413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-07e7f73d-a206-4e25-9bdf-06b234ed9921 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Calling method '.version_select at 0x7f72c0d46b80>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:56.979612 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-07e7f73d-a206-4e25-9bdf-06b234ed9921 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a" status: 200 len: 353 microversion: 2.1 time: 0.053096 Sep 30 09:47:56.979612 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 121/235] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:56 2022] GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a => generated 353 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:47:57.583952 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-642890262", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "networks": [{"uuid": "19f705b3-87b0-406c-8a82-287369b1f01f"}]}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:47:57.647395 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] validate_networks() for [('19f705b3-87b0-406c-8a82-287369b1f01f', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:47:57.873028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:57.874440 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:57.882627 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:57.883183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:57.883624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:57.884559 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:57.885026 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:57.885453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:58.185134 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:47:58.185663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:47:58.186809 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:47:58.187292 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:47:58.188024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:47:58.202577 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:58.210607 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7992e763-02cc-4aea-b7cb-799215c91ad4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.342062 Sep 30 09:47:58.215078 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 122/236] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:57 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:58.232210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:58.233120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:58.233413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9431dee5-3fc1-44e1-8044-3aa78c41c187 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Calling method '.version_select at 0x7f72c0ff45e0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:58.234525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:58.235942 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:58.236511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:58.237606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:58.266366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-9431dee5-3fc1-44e1-8044-3aa78c41c187 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] HTTP exception thrown: Volume 3fe73289-fca5-4843-9546-cc4d4fb6d73a could not be found. Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9431dee5-3fc1-44e1-8044-3aa78c41c187 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Request method failure captured: Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a HTTP/1.1 Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrtK38UsEThqntq0dq4tSXLNkmVU4vIgEMgcDDmGaKYovN9VMjSjwnmSiWZ9FbdRbwVo2pdXldpedxPo7sy9Kz8Ptsb7WasqnlCHoAnC2UNHc57VA2vmsxCWhPo0mor4PLj1xTRwp_Bv-7zNChUPb8PSRDb9Q4i5QzsixuZWfxFPxxI Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 828649edae4b4cbaa1176341d5d933b0 Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-VolumesGetTestJSON-1166706591 Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:47:58.269265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/828649edae4b4cbaa1176341d5d933b0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/828649edae4b4cbaa1176341d5d933b0"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/828649edae4b4cbaa1176341d5d933b0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_828649edae4b4cbaa1176341d5d933b0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-VolumesGetTestJSON-1166706591 Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 828649edae4b4cbaa1176341d5d933b0 Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-VolumesGetTestJSON-1166706591 Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-VolumesGetTestJSON-1166706591-project Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 99ccdcda76aa4458831660357f1f8956 Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-VolumesGetTestJSON-1166706591-project Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Volume 3fe73289-fca5-4843-9546-cc4d4fb6d73a could not be found. Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '3fe73289-fca5-4843-9546-cc4d4fb6d73a'} Sep 30 09:47:58.270855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 3fe73289-fca5-4843-9546-cc4d4fb6d73a could not be found. (HTTP 404) (Request-ID: req-0b1bbb7f-e2d4-4e48-b4b3-301baab17b0c) Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:58.271848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 09:47:58.273119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 3fe73289-fca5-4843-9546-cc4d4fb6d73a could not be found. Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 30 09:47:58.274516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 3fe73289-fca5-4843-9546-cc4d4fb6d73a could not be found. Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 3fe73289-fca5-4843-9546-cc4d4fb6d73a could not be found. Sep 30 09:47:58.275822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:47:58.277849 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9431dee5-3fc1-44e1-8044-3aa78c41c187 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] Returning 404 to user: Volume 3fe73289-fca5-4843-9546-cc4d4fb6d73a could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:47:58.279071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9431dee5-3fc1-44e1-8044-3aa78c41c187 tempest-VolumesGetTestJSON-1166706591 tempest-VolumesGetTestJSON-1166706591-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a" status: 404 len: 109 microversion: 2.1 time: 0.062838 Sep 30 09:47:58.283381 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 123/237] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 09:47:58 2022] GET /compute/v2.1/os-volumes/3fe73289-fca5-4843-9546-cc4d4fb6d73a => generated 109 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:47:58.284898 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Getting quotas for project c1eb9ddbc0a84a308ceffeae5198f229. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:47:58.295574 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Getting quotas for user 587169e32cf64fe5b684e4b2ce0701a8 and project c1eb9ddbc0a84a308ceffeae5198f229. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:47:58.313695 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:47:58.314367 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:47:58.498943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0696042f-7f9a-42c8-ba1a-93b42388e055 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.919904 Sep 30 09:47:58.500373 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 115/238] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:47:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 922 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:47:58.511024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2df702a0-a759-4340-81c8-f6af65e64e74 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:58.512205 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2df702a0-a759-4340-81c8-f6af65e64e74 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:58.581969 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2df702a0-a759-4340-81c8-f6af65e64e74 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:58.600871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2df702a0-a759-4340-81c8-f6af65e64e74 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1213 microversion: 2.1 time: 0.093454 Sep 30 09:47:58.601574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 124/239] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:58 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1213 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:59.238109 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:59.239782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:59.250025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:59.250527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:59.250995 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:59.251702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:47:59.252172 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:47:59.257309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:47:59.500036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:59.507863 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1ce316ff-2827-4095-8ac1-74888b94dfd4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.274700 Sep 30 09:47:59.510951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 116/240] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:59 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:47:59.630664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a5180bf0-70d7-4a3a-9119-65bccae39edc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:47:59.630664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a5180bf0-70d7-4a3a-9119-65bccae39edc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:47:59.694062 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a5180bf0-70d7-4a3a-9119-65bccae39edc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:47:59.703147 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a5180bf0-70d7-4a3a-9119-65bccae39edc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1213 microversion: 2.1 time: 0.082057 Sep 30 09:47:59.703727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 125/241] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:47:59 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:00.534334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:00.535583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:00.542208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:00.543044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:00.543531 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:00.544500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:00.546882 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:00.550232 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:00.727568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:00.728749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:00.735608 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:00.736654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:00.737305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:00.738200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:00.742402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:00.742902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:00.825978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:00.833963 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ca855529-aa35-4ea7-bf7a-7785a4ce8b7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.303426 Sep 30 09:48:00.834945 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 117/242] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:00 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:00.844392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:00.854973 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bc260b65-ed5d-4701-b263-e719e398c07c tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1217 microversion: 2.1 time: 0.131794 Sep 30 09:48:00.856029 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 126/243] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:00 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1217 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:01.856235 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:01.856792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:01.868396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:01.869784 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:01.870452 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:01.871512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:01.872189 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:01.874037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:01.881581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:01.882927 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:01.890089 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:01.890712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:01.891395 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:01.892248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:01.892743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:01.893287 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:02.372329 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:02.383400 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e0d67c29-d3ba-497a-a1da-3c385c5ed7ae tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1265 microversion: 2.1 time: 0.506911 Sep 30 09:48:02.385098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 127/244] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:01 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1265 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:02.563021 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:02.572355 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-07ed1e9a-5564-4993-91a8-4ee75734c0e2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.719903 Sep 30 09:48:02.572997 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 118/245] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:01 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:03.413151 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:03.413767 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:03.420742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:03.421460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:03.421899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:03.422620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:03.423122 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:03.423546 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:03.510540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:03.519139 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-16867ac7-17b1-4cc6-a924-03ab4f0bedec tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1283 microversion: 2.1 time: 0.109929 Sep 30 09:48:03.520347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 128/246] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:03 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1283 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:03.599160 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:03.600359 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:03.607086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:03.608376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:03.608812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:03.610918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:03.610918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:03.610918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:03.753919 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:03.761899 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f00e8648-40d0-4674-863d-f13bbfb2bc64 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.170524 Sep 30 09:48:03.762772 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 119/247] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:03 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:04.543052 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:04.543681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:04.550978 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:04.551768 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:04.552479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:04.553695 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:04.554366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:04.555031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:04.670871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:04.678453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0a4811b8-6c80-48a8-949b-f2e5d3a24b61 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1283 microversion: 2.1 time: 0.139250 Sep 30 09:48:04.679391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 129/248] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:04 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1283 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:04.785131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:04.785746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:04.792568 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:04.793055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:04.793488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:04.794185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:04.794651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:04.795753 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:05.008501 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:05.016161 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-30bafa87-718c-43c8-a4b7-bda60b4dec73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.234714 Sep 30 09:48:05.016725 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 120/249] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:04 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:05.682825 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:05.687873 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:05.697892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:05.698678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:05.699322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:05.700231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:05.700838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:05.701990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:05.703184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:05.713661 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:05.722759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:05.723252 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:05.724228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:05.724942 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:05.725391 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:05.725952 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:05.731964 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1929cbd0-b239-4c02-8269-ef4c90449e8a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/ips" status: 200 len: 150 microversion: 2.1 time: 0.053918 Sep 30 09:48:05.732601 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 130/250] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 09:48:05 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/ips => generated 150 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:05.800508 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:05.816227 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b525927c-2116-4101-9a71-033f1c7766e8 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1283 microversion: 2.1 time: 0.112793 Sep 30 09:48:05.816227 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 121/251] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:05 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1283 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:06.045510 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:06.046131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:06.052720 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:06.053436 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:06.054110 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:06.055074 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:06.055608 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:06.056150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:06.454516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:06.465885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8a3044b5-46ec-4c86-a3b3-4e22485ed48c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.429072 Sep 30 09:48:06.466835 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 131/252] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:06 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:06.837434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:06.838784 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:06.849434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:06.849434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:06.849715 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:06.850560 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:06.851173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:06.851738 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:06.934651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:06.948010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c0df09b3-8dbc-4c66-8ff4-09b3f7fb84d4 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1271 microversion: 2.1 time: 0.113346 Sep 30 09:48:06.950323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 122/253] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:06 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1271 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:07.493836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:07.495204 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:07.502113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:07.503098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:07.503745 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:07.504561 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:07.505154 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:07.505792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:07.670302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:07.676572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5b9af4ec-f791-4f8d-be9e-35167d93ba8a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.190847 Sep 30 09:48:07.677144 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 132/254] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:07 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:07.973223 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:07.976560 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:07.985800 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:07.986309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:07.986789 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:07.987509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:07.988000 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:07.988428 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:08.260685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=2 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:08.290383 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:08.299859 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9936b972-04ba-43d0-92a7-9bd941315379 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.330439 Sep 30 09:48:08.310825 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 123/255] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:07 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:08.699906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:08.700518 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:08.710231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:08.710979 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:08.711422 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:08.712119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:08.712567 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:08.712998 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:08.941136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:08.949845 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c4923e5d-b9b5-4ec9-8ae0-f014e77667e8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.253753 Sep 30 09:48:08.951660 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 133/256] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:08 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:09.329054 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:09.329779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:09.336573 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=6 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:09.338669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:09.342105 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:09.342739 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:09.343182 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:09.343890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:09.345520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:09.345981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:09.508904 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:09.517422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8ebf441a-4804-4d96-ab96-2b3bc8f6cd08 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.195195 Sep 30 09:48:09.517991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 124/257] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:09 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:09.977004 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:09.977661 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:09.982917 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=4 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:09.988050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:09.988617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:09.989689 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:09.990785 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:09.991309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:09.991787 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:10.183554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:10.196132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-174818d2-10b6-4a83-91c3-17b01335db84 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.225893 Sep 30 09:48:10.197232 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 134/258] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:09 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:10.548932 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:10.548932 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:10.578545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:10.579373 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:10.580005 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:10.580979 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:10.582445 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:10.585811 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:10.788534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:10.799488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6603abd8-cd04-4fc8-b81a-7d8707cf2d46 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.260193 Sep 30 09:48:10.800048 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 125/259] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:10 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:11.221705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:11.223241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:11.230017 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:11.230516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:11.231192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:11.231842 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:11.232372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:11.232805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:11.444403 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:11.447761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=100 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:11.452308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-786ee76e-72de-4ccb-8b8e-f60914f330fd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.235257 Sep 30 09:48:11.452954 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 135/260] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:11 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:11.825094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:11.826836 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:11.834025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:11.835023 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:11.835449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:11.836717 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:11.837170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:11.837586 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:11.991548 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:12.001454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d4271ebc-a453-4c6a-a7a2-ec6f0f638afe tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.180459 Sep 30 09:48:12.002027 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 126/261] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:11 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:12.477765 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:12.479910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:12.490904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:12.492012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:12.492573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:12.493736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:12.494348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:12.495266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:13.006378 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:13.126495 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bf2ad722-22d5-456d-b94a-91620f973ca5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.543876 Sep 30 09:48:13.126610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:13.126610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:13.127216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 136/262] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:12 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:13.127298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:13.127298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:13.127298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:13.127298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:13.127569 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:13.127569 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:13.127569 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:13.359089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:13.359089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c2b73051-eb80-47ac-9dad-cdb89a476fa3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.293568 Sep 30 09:48:13.359089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 127/263] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:13 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:13.398742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c3971c02-ab43-47a0-bad0-be731ce4d4be"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:13.398742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:13.398742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:13.398742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:13.399738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:13.399738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:13.399738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:13.399738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:14.151484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:14.151484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:14.161416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:14.162087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:14.162422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:14.163170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:14.163815 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:14.165433 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:14.848590 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-02a99717-29e6-45ac-8486-1631b75f70b3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.465801 Sep 30 09:48:14.855101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 137/264] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:48:13 2022] POST /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments => generated 194 bytes in 1468 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:14.861316 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:14.875509 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:14.891914 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=47 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:14.895178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:14.899096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:14.907305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:14.909567 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:14.911280 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:14.912226 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:14.924899 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:14.942796 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-02cec66f-6051-45fa-9627-a7f559155f7d tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.802096 Sep 30 09:48:14.943385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 128/265] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:14 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 803 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:15.250640 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-d51100f3-0f42-495c-977c-5b77d78f5392 req-bfda5534-0f83-41fa-9e14-d184f4420175 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9208352-ebfe-4580-abf1-4e55bb98a5a3", "tag": "789f9272-6c06-496f-83bd-184cd3013e84"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:15.270185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d51100f3-0f42-495c-977c-5b77d78f5392 req-bfda5534-0f83-41fa-9e14-d184f4420175 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:15.282692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d51100f3-0f42-495c-977c-5b77d78f5392 req-bfda5534-0f83-41fa-9e14-d184f4420175 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:15.298042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d51100f3-0f42-495c-977c-5b77d78f5392 req-bfda5534-0f83-41fa-9e14-d184f4420175 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:15.386138 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-d51100f3-0f42-495c-977c-5b77d78f5392 req-bfda5534-0f83-41fa-9e14-d184f4420175 service nova] Creating event network-changed:789f9272-6c06-496f-83bd-184cd3013e84 for instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 on hv-cinder-maste Sep 30 09:48:15.420836 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-d51100f3-0f42-495c-977c-5b77d78f5392 req-bfda5534-0f83-41fa-9e14-d184f4420175 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.170475 Sep 30 09:48:15.425381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 129/266] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:15.562294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:15.570657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-6a174cc2-151f-4743-94f1-b75b7507f330 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.719132 Sep 30 09:48:15.576019 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 138/267] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:14 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:15.969239 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:15.969986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:15.988260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:15.988752 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:15.989188 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:15.989890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:15.990337 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:15.990833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:16.598045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:16.598819 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:16.605174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:16.611032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:16.611032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:16.611032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:16.611032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:16.611428 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:16.699047 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:16.708441 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0bb4fd4e-3d66-4895-8316-65078a2f30f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1501 microversion: 2.1 time: 0.743506 Sep 30 09:48:16.712071 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 130/268] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:15 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1501 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:16.742516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4783d022-f666-47e5-9533-5dc54949f01f"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:16.745385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:16.757024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:16.758624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:16.761466 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:16.761466 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:16.772232 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:16.772969 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:16.827132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] HTTP exception thrown: Cannot 'attach_volume' instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec while it is in vm_state rescued Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Request method failure captured: Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: POST /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec/os-volume_attachments HTTP/1.1 Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 74 Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrTlXnTtrSqp1V_x1W1vnIRpWmkBjrFT8cNypZpVQr27Bt1ba58D9-pK8ZLi7HjxFCHn6nlNUOFaxuG8SOFGbJyOl9-u5Y9sbOG4I2nKOR06Qh9xhjqVXK1QVMfkONcO4rT4iNkpR9PqGmjOgNcaMeH-UkdqodqHrtoefVyVzzrL0A Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:48:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 7ad21da05fc246d489b0afa641f20216 Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {"volumeAttachment": {"volumeId": "4783d022-f666-47e5-9533-5dc54949f01f"}} Sep 30 09:48:16.831362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Cannot 'attach_volume' instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec while it is in vm_state rescued Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'server_id': 'fd8e4eb8-1261-4abf-82ec-e1d8afda21ec', 'body': {'volumeAttachment': {'volumeId': '4783d022-f666-47e5-9533-5dc54949f01f'}}} Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:48:16.832155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec while it is in vm_state rescued Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:48:16.833873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:48:16.836754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:48:16.836754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:48:16.836754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec while it is in vm_state rescued Sep 30 09:48:16.836754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:48:16.862267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Returning 409 to user: Cannot 'attach_volume' instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec while it is in vm_state rescued {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:48:16.864516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c8711046-491a-4fff-9cfd-c0da239dd73f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.127395 Sep 30 09:48:16.881958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 131/269] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:48:16 2022] POST /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec/os-volume_attachments => generated 152 bytes in 145 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 30 09:48:16.906474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:16.926729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:16.974564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:16.994361 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:16.995412 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:16.996378 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:17.000986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:17.001731 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:17.152284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:17.162069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-18386cda-cce6-4b3a-862e-9dc3af9a30e7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.567892 Sep 30 09:48:17.162952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 139/270] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:16 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:17.264114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b1671800-d3f6-4256-8018-f605e980aa5a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec/action" status: 202 len: 0 microversion: 2.1 time: 0.375677 Sep 30 09:48:17.264114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 132/271] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:48:16 2022] POST /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec/action => generated 0 bytes in 377 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:48:17.269030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:17.269618 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:17.276881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:17.277647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:17.278279 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:17.279223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:17.279834 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:17.280396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:17.589101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:17.597340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fba70b57-2c50-4644-b54f-b5e390fa4105 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1509 microversion: 2.1 time: 0.332060 Sep 30 09:48:17.610619 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 140/272] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:17 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1509 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:18.194536 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:18.207544 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:18.225233 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:18.226002 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:18.227606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:18.229178 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:18.232809 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:18.233433 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:18.527371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:18.536981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6ee4d40e-c60d-4ce9-ad50-afc76790b8bf tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.354363 Sep 30 09:48:18.538057 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 133/273] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:18 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:18.626453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:18.627081 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:18.644963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:18.645749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:18.646192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:18.646958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:18.647413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:18.647826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:18.901549 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:18.910414 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1148895b-c225-4da7-8cbd-77d4583d4ffb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1509 microversion: 2.1 time: 0.288533 Sep 30 09:48:18.914960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 141/274] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:18 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1509 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:19.564231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:19.569995 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:19.572164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:19.572645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:19.573099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:19.573798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:19.574241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:19.574702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:19.754338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:19.797842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ddd93635-89d6-42a2-8faa-35a4ea995398 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.235683 Sep 30 09:48:19.800578 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 134/275] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:19 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:19.939463 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:19.944658 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:19.955172 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:19.958997 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:19.959689 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:19.960477 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:19.963238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:19.964205 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:20.148571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:20.157361 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b704c69d-f01c-4a0c-b3c6-a4ccb08cdfe9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1509 microversion: 2.1 time: 0.222045 Sep 30 09:48:20.158127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 142/276] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:19 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1509 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:20.849662 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:20.853479 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:20.860197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:20.863553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:20.863553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:20.863553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:20.863553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:20.863933 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:21.037069 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:21.045860 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-235ac881-8a09-4520-ab14-859e1269e67a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.219261 Sep 30 09:48:21.061046 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 135/277] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:20 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:21.184043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:21.184888 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:21.191428 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:21.192838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:21.193445 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:21.194327 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:21.197958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:21.199029 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:21.357510 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:21.365996 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4000d945-257f-44b2-9496-6df3082c28c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1509 microversion: 2.1 time: 0.187094 Sep 30 09:48:21.366576 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 143/278] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:21 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1509 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:22.070959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:22.072209 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:22.080334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:22.081399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:22.081993 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:22.083344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:22.083955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:22.084599 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:22.254480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:22.280473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ceed361d-c02c-4559-85f3-7c2a9fa275b7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.202354 Sep 30 09:48:22.280473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 136/279] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:22 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:22.395625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:22.396307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:22.415162 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:22.419555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:22.420241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:22.421192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:22.422056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:22.422759 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:22.648187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:22.661760 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-66a637ed-80c1-4e42-87de-794bb15cdf75 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1509 microversion: 2.1 time: 0.271434 Sep 30 09:48:22.665990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 144/280] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:22 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1509 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:22.905649 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-91b7d3c0-6a43-4d38-9453-6f4bb14db282 req-a7f39f54-a0b4-49ab-a4a6-a0938111646c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9208352-ebfe-4580-abf1-4e55bb98a5a3", "name": "network-vif-plugged", "status": "completed", "tag": "789f9272-6c06-496f-83bd-184cd3013e84"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:22.943424 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-91b7d3c0-6a43-4d38-9453-6f4bb14db282 req-a7f39f54-a0b4-49ab-a4a6-a0938111646c service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:22.943908 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-91b7d3c0-6a43-4d38-9453-6f4bb14db282 req-a7f39f54-a0b4-49ab-a4a6-a0938111646c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:22.944381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-91b7d3c0-6a43-4d38-9453-6f4bb14db282 req-a7f39f54-a0b4-49ab-a4a6-a0938111646c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:22.958522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-91b7d3c0-6a43-4d38-9453-6f4bb14db282 req-a7f39f54-a0b4-49ab-a4a6-a0938111646c service nova] Creating event network-vif-plugged:789f9272-6c06-496f-83bd-184cd3013e84 for instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 on hv-cinder-maste Sep 30 09:48:22.964363 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-91b7d3c0-6a43-4d38-9453-6f4bb14db282 req-a7f39f54-a0b4-49ab-a4a6-a0938111646c service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065452 Sep 30 09:48:22.965107 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 137/281] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:23.303707 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:23.307698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:23.313245 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:23.314346 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:23.314967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:23.316416 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:23.317033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:23.317753 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:23.546398 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:23.556984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-51904e9d-6b6c-4ee1-b81f-f2eb8077321b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.265472 Sep 30 09:48:23.557558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 145/282] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:23 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:23.686148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:23.687404 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:23.694882 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:23.695518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:23.696264 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:23.697170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:23.697793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:23.698389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:23.873796 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:23.882064 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-074aca7d-0135-423a-bac6-a715c04857a0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1515 microversion: 2.1 time: 0.200768 Sep 30 09:48:23.884144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 138/283] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:23 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1515 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:24.581431 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:24.582030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:24.593293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:24.595114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:24.597001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:24.599247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:24.602175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:24.602814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:24.845322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:24.855043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e6e5b2ac-3daf-4000-800d-f7b433881327 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.278270 Sep 30 09:48:24.857799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 146/284] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:24 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:25.671050 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b43c24fd-57e5-47db-ae19-390a32e2313c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: .version_select at 0x7f72c0c6aaf0>, body: {"keypair": {"name": "tempest-keypair-2192127", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFMyLD4VoIwWHPRvo0Ekp+mSr8wudGR7JJK6/iu6c5/F7EELjx+Bw+jbe6e8p8l0+eCtOeHMtdEyopJcrLqIqVy8NRhsHzHio75c1GWfnf4B12XW02cEUezlXuUbZONDnA=="}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:25.695286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-b43c24fd-57e5-47db-ae19-390a32e2313c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for project 86e6b0501cb94028be07bddd2182638b. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:25.695286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-b43c24fd-57e5-47db-ae19-390a32e2313c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for user 8c4ced79c3ff4842b30b472acb555cd5 and project 86e6b0501cb94028be07bddd2182638b. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:25.883200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:25.883803 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:25.890918 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:25.891402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:25.891836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:25.892521 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:25.892972 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:25.893391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:25.915380 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b43c24fd-57e5-47db-ae19-390a32e2313c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 383 microversion: 2.1 time: 0.247165 Sep 30 09:48:25.920382 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 139/285] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:48:25 2022] POST /compute/v2.1/os-keypairs => generated 383 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:26.336736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:26.345276 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e14aca62-feb0-416d-a53f-9a3a8cd0aa10 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1313 microversion: 2.1 time: 0.465817 Sep 30 09:48:26.345846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 147/286] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:25 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1313 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:26.846490 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0a755118-b59f-4847-b655-755e362ec499 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'create', calling method: .version_select at 0x7f72c0e91ee0>, body: {"keypair": {"name": "tempest-keypair-1268259575", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDb1Q2TNPM7SizUGTb0zX7CzS4sBc22uX8uqJTAmC6MQIfSlpMfLwgJmhZMTrSuZ5OzjeTLrceJ1UOTnjk/b4/9z1qQbtans+dBQNG5UMF1RN8vGn+tE1lzWJGoHzLz2RQ=="}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:26.864301 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-0a755118-b59f-4847-b655-755e362ec499 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Getting quotas for project f1c0948ddb014e6f93d6069bb5203890. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:26.868635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-0a755118-b59f-4847-b655-755e362ec499 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Getting quotas for user 7ad21da05fc246d489b0afa641f20216 and project f1c0948ddb014e6f93d6069bb5203890. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:26.940880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-27db0fcc-5580-4022-9d0e-22538602dfc7 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Action: 'create', calling method: .version_select at 0x7f72c0e285e0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1896860384", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGGQKSSf5JF0fBjvrzntOO8QQdYeSEEh2+gqAv1uBJGPtnN+LcY8GmSu7H8N4oXvkJoEtwRDkSFG+spwqSryTa62DaHw8lx3QNVxJtvI6iyV5VnCRJtgmvHW4riSRMH4zQ=="}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:26.957446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-27db0fcc-5580-4022-9d0e-22538602dfc7 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Getting quotas for project 0be64f8f1d4b452e84a7bcd64f6100d1. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:26.963960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-27db0fcc-5580-4022-9d0e-22538602dfc7 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Getting quotas for user 4d46ec34caf74ea7b7c8f836c2f7ea6c and project 0be64f8f1d4b452e84a7bcd64f6100d1. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:26.994621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0a755118-b59f-4847-b655-755e362ec499 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.152236 Sep 30 09:48:26.995193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 140/287] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:48:26 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:48:27.045584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-27db0fcc-5580-4022-9d0e-22538602dfc7 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.108367 Sep 30 09:48:27.046180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 148/288] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:48:26 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:48:27.164599 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2106840121", "imageRef": "6e7968d5-3ba8-4d03-8220-636c50a8e90a", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1896860384", "networks": [{"uuid": "5fc7a94e-ea2f-40e2-8326-488bd0c4fbe7"}]}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:27.220715 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] validate_networks() for [('5fc7a94e-ea2f-40e2-8326-488bd0c4fbe7', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:48:27.369771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:27.371963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:27.381836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:27.382791 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:27.383437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:27.384341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:27.384876 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:27.385243 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:27.659965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:27.668006 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4101f47e-e866-4fa0-94d8-89ed690ca770 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1496 microversion: 2.1 time: 0.302511 Sep 30 09:48:27.669643 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 149/289] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:27 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1496 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:27.737165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:48:27.738091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:48:27.738728 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:48:27.739234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:48:27.740225 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:48:27.749718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:27.750286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:27.750820 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:27.751606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:27.752121 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:27.752601 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:27.777222 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Getting quotas for project 0be64f8f1d4b452e84a7bcd64f6100d1. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:27.782952 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Getting quotas for user 4d46ec34caf74ea7b7c8f836c2f7ea6c and project 0be64f8f1d4b452e84a7bcd64f6100d1. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:27.796263 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:48:27.798648 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='6e7968d5-3ba8-4d03-8220-636c50a8e90a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:48:27.969371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a828ef39-483f-4ca3-adfb-0eabe27c7475 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.812131 Sep 30 09:48:27.980957 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8a6679b5-836e-4b12-83e8-87219ce3230e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:27.981571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8a6679b5-836e-4b12-83e8-87219ce3230e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:27.986915 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 141/290] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:48:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 831 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:48:28.059119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8a6679b5-836e-4b12-83e8-87219ce3230e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:28.067067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8a6679b5-836e-4b12-83e8-87219ce3230e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1259 microversion: 2.1 time: 0.090040 Sep 30 09:48:28.070859 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 150/291] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:27 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1259 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:28.801285 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:28.802011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:28.809669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:28.810218 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:28.812264 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:28.812264 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:28.812264 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:28.812756 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:28.841286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:48:29.095097 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-574e4eda-5141-4b4c-bbd0-096c6dc7c082 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:29.096271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-574e4eda-5141-4b4c-bbd0-096c6dc7c082 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:29.181305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-574e4eda-5141-4b4c-bbd0-096c6dc7c082 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:29.190317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-574e4eda-5141-4b4c-bbd0-096c6dc7c082 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1259 microversion: 2.1 time: 0.100851 Sep 30 09:48:29.206517 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 151/292] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:29 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1259 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:29.273526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8336ca2c-1d9a-4501-8688-803045af9c0a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "DELETE /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 204 len: 0 microversion: 2.1 time: 0.476332 Sep 30 09:48:29.274219 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 142/293] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:48:28 2022] DELETE /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:48:29.283791 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:29.291486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:29.291486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:29.291486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:29.293731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:29.294706 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:29.295186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:29.296275 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:29.417129 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-315957042", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d038a068-f643-497f-8e6f-7d6e8fb5892b"}], "security_groups": [{"name": "tempest-securitygroup--703128740"}], "key_name": "tempest-keypair-2192127", "user_data": "}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:29.483211 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:29.495405 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-eda839fc-a741-40b0-a615-076fc805aaed tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1502 microversion: 2.1 time: 0.215686 Sep 30 09:48:29.504718 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 152/294] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:29 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1502 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:29.577441 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] validate_networks() for [('d038a068-f643-497f-8e6f-7d6e8fb5892b', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:48:30.025675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1884710726", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "069f8527-f158-4f06-809a-7a507bf3fc84"}], "security_groups": [{"name": "tempest-securitygroup--2041892264"}], "key_name": "tempest-keypair-1268259575", "user_data": "}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:30.213515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] validate_networks() for [('069f8527-f158-4f06-809a-7a507bf3fc84', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:48:30.220681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:48:30.221096 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:48:30.221560 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:48:30.221965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:48:30.222362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:48:30.233473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:30.234180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:30.234833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:30.235805 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:30.240249 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:30.240249 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:30.259349 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for project 86e6b0501cb94028be07bddd2182638b. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:30.274152 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for user 8c4ced79c3ff4842b30b472acb555cd5 and project 86e6b0501cb94028be07bddd2182638b. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:30.295556 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:48:30.297980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:48:30.489746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-bebcba81-f710-4832-9683-bd97984ca1d2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.077120 Sep 30 09:48:30.490437 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 143/295] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:48:29 2022] POST /compute/v2.1/servers => generated 401 bytes in 1079 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:48:30.503619 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:30.504503 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:30.528194 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:30.528612 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:30.529037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:30.529732 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:30.530174 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:30.530624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:30.661196 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:30.673678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2a1725db-81d4-4177-a36e-29dd4c8fee70 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1263 microversion: 2.1 time: 0.181431 Sep 30 09:48:30.675117 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 144/296] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:30 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1263 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:30.680375 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f9d57ddf-62b7-4426-9fba-419d9841e90d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:30.681172 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f9d57ddf-62b7-4426-9fba-419d9841e90d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:30.685299 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=8,nova_api:UPDATE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:30.704021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:48:30.704450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:48:30.704920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:48:30.705391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:48:30.705999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:48:30.717235 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:30.717928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:30.718526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:30.719445 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:30.720053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:30.720652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:30.743330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Getting quotas for project f1c0948ddb014e6f93d6069bb5203890. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:30.752439 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Getting quotas for user 7ad21da05fc246d489b0afa641f20216 and project f1c0948ddb014e6f93d6069bb5203890. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:30.762038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:48:30.762887 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f9d57ddf-62b7-4426-9fba-419d9841e90d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:30.770715 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f9d57ddf-62b7-4426-9fba-419d9841e90d tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1238 microversion: 2.1 time: 0.094363 Sep 30 09:48:30.771776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 145/297] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:30 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1238 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:30.772686 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:48:30.776923 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:30.778918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:30.797920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:30.798422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:30.799147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:30.799821 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:30.800381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:30.800748 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:30.929892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c56b01cf-acfe-4f69-b99f-e6113a963d7e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.909325 Sep 30 09:48:30.931104 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 153/298] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:48:30 2022] POST /compute/v2.1/servers => generated 409 bytes in 911 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 30 09:48:30.952151 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9511c3ac-63f4-43a3-a3ce-8ed4e5c7d5ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:30.952932 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9511c3ac-63f4-43a3-a3ce-8ed4e5c7d5ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:31.107128 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9511c3ac-63f4-43a3-a3ce-8ed4e5c7d5ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:31.112638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:31.122869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9511c3ac-63f4-43a3-a3ce-8ed4e5c7d5ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1246 microversion: 2.1 time: 0.178050 Sep 30 09:48:31.123456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 154/299] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:30 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1246 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:31.124113 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b16a5c11-5d6a-4e9d-8757-f537be698876 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1502 microversion: 2.1 time: 0.350728 Sep 30 09:48:31.129123 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 146/300] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:30 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1502 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:31.704457 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:31.705084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-bab36e8c-e904-4e38-b092-1cc418e82b2b req-3e0ee497-be4e-41ef-8717-84b47c33ecd0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9208352-ebfe-4580-abf1-4e55bb98a5a3", "name": "network-vif-deleted", "tag": "789f9272-6c06-496f-83bd-184cd3013e84"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:31.706384 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:31.714578 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:31.715814 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:31.716403 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:31.717371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:31.717981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:31.718803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:31.722557 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:31.726836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-bab36e8c-e904-4e38-b092-1cc418e82b2b req-3e0ee497-be4e-41ef-8717-84b47c33ecd0 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:31.727387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-bab36e8c-e904-4e38-b092-1cc418e82b2b req-3e0ee497-be4e-41ef-8717-84b47c33ecd0 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:31.727837 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-bab36e8c-e904-4e38-b092-1cc418e82b2b req-3e0ee497-be4e-41ef-8717-84b47c33ecd0 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:31.765620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-bab36e8c-e904-4e38-b092-1cc418e82b2b req-3e0ee497-be4e-41ef-8717-84b47c33ecd0 service nova] Creating event network-vif-deleted:789f9272-6c06-496f-83bd-184cd3013e84 for instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 on hv-cinder-maste Sep 30 09:48:31.775185 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-bab36e8c-e904-4e38-b092-1cc418e82b2b req-3e0ee497-be4e-41ef-8717-84b47c33ecd0 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075517 Sep 30 09:48:31.775730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 155/301] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:48:31.797189 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a9171e2a-aa33-415b-b66b-fe49956e1541 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:31.798175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a9171e2a-aa33-415b-b66b-fe49956e1541 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:31.812855 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:31.820556 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b5b77276-77dd-4a10-839e-f0e5ebf9ba57 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1311 microversion: 2.1 time: 0.120391 Sep 30 09:48:31.821120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 147/302] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:31 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1311 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:31.868032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a9171e2a-aa33-415b-b66b-fe49956e1541 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:31.875899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a9171e2a-aa33-415b-b66b-fe49956e1541 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1238 microversion: 2.1 time: 0.083302 Sep 30 09:48:31.876494 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 156/303] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:31 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1238 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:32.148368 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:32.148989 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:32.153642 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:32.154578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:32.157540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:32.161115 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:32.161488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:32.162489 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:32.163033 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:32.165420 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:32.165540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:32.165540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:32.165540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:32.165540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:32.165874 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:32.166006 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:32.190035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:32.254689 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:32.267654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:32.271163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-cd67ada4-6879-49b7-9142-e01073e166ce tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1250 microversion: 2.1 time: 0.123338 Sep 30 09:48:32.272085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 148/304] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:32 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1250 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:32.292267 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3bcd9e02-6d94-4776-9b06-b9cb90822d42 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1298 microversion: 2.1 time: 0.141619 Sep 30 09:48:32.292267 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 157/305] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:32 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1298 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:32.844682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:32.845996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:32.860495 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:32.861232 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:32.861932 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:32.863369 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:32.865273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:32.867882 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:32.899161 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:32.899765 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:32.918581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:32.919103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:32.919538 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:32.920251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:32.920699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:32.921147 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:33.044159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:33.044584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:33.069156 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-68399220-0938-467a-87fe-47f57cdea68d tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1317 microversion: 2.1 time: 0.225805 Sep 30 09:48:33.070828 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 149/306] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:32 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1317 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:33.075073 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9be725c4-cf21-4ebb-9f26-23a503461346 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1234 microversion: 2.1 time: 0.178587 Sep 30 09:48:33.075841 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 158/307] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:32 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1234 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:33.293959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:33.294564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:33.301331 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:33.318957 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:33.319573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:33.325624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.024s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:33.326102 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:33.326865 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:33.328719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:33.329498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:33.330086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:33.335153 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:33.335845 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:33.336741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:33.337363 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:33.343639 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:33.402107 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:33.402862 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:33.403543 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:33.404131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lazy-loading 'fault' on Instance uuid d9208352-ebfe-4580-abf1-4e55bb98a5a3 {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:48:33.560459 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:33.578997 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c1606cf9-b4f8-4d68-a97f-7f68cbfddfba tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1242 microversion: 2.1 time: 0.289208 Sep 30 09:48:33.588914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 150/308] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:33 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1242 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:33.633439 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:33.641729 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ceb2167d-e188-453f-80a7-7af9512c5dc6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 200 len: 1303 microversion: 2.1 time: 0.327428 Sep 30 09:48:33.651446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 159/309] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:33 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 1303 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:34.116398 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:34.117023 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:34.120438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:34.121619 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:34.185823 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:34.191114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:34.191953 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:34.211011 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:34.211121 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:34.211293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:34.211293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:34.211474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:34.211563 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:34.211650 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:34.211741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:34.211885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:34.750137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:34.759454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c620a9d0-8645-4d5b-8a0e-d57b99ed3e20 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1290 microversion: 2.1 time: 0.659921 Sep 30 09:48:34.764733 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 151/310] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:34 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1290 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:34.786000 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:34.787782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:34.802260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:34.804240 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:34.805173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:34.806260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:34.809359 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:34.810021 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:35.283173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:35.287241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:35.291513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8b0ecb13-8a8e-43a8-9e67-5d9e6642b250 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 1.184973 Sep 30 09:48:35.292100 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 160/311] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:34 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 1187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:35.298835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9b212966-6381-4483-8ef2-b613c2050d43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1298 microversion: 2.1 time: 0.532276 Sep 30 09:48:35.306636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 152/312] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:34 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1298 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:35.337305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:35.337928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d9208352-ebfe-4580-abf1-4e55bb98a5a3] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:35.360050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:35.361111 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:35.363457 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:35.364379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:35.365004 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:35.365604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:35.391323 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] HTTP exception thrown: Instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 could not be found. Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Request method failure captured: Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 HTTP/1.1 Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrtKwaffcg461ol3LZNOL1Yy5_5CkyV3Y_EfWjrrUVQYdeYaK3kNuLZ0_ojEdr86yPbtav26zSgCHXWPEXcJLBKFISfTCeemF7xQPK1GMOickHtS25WzWVMeTFpKuqRmN1J3MV3QbIPSXgL1kos5fSqeoMbC-ZnEvnei6T9RU8SCN78 Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: c1eb9ddbc0a84a308ceffeae5198f229 Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-VolumesActionsTest-1876305362 Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:48:35.396654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/c1eb9ddbc0a84a308ceffeae5198f229"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/c1eb9ddbc0a84a308ceffeae5198f229"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/c1eb9ddbc0a84a308ceffeae5198f229"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_c1eb9ddbc0a84a308ceffeae5198f229"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-VolumesActionsTest-1876305362 Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: c1eb9ddbc0a84a308ceffeae5198f229 Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-VolumesActionsTest-1876305362 Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-VolumesActionsTest-1876305362-project Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 587169e32cf64fe5b684e4b2ce0701a8 Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-VolumesActionsTest-1876305362-project Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 could not be found. Sep 30 09:48:35.398761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': 'd9208352-ebfe-4580-abf1-4e55bb98a5a3'} Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 could not be found. Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:48:35.399705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 could not be found. Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 could not be found. Sep 30 09:48:35.401296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:48:35.404548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Returning 404 to user: Instance d9208352-ebfe-4580-abf1-4e55bb98a5a3 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:48:35.435565 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9d65ae47-6181-4afb-a336-24dfcc430b9a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3" status: 404 len: 111 microversion: 2.1 time: 0.142135 Sep 30 09:48:35.444624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 161/313] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:35 2022] GET /compute/v2.1/servers/d9208352-ebfe-4580-abf1-4e55bb98a5a3 => generated 111 bytes in 152 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:48:35.775095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1847428238", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "networks": [{"uuid": "19f705b3-87b0-406c-8a82-287369b1f01f"}]}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:35.786873 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:35.787675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:35.798513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:35.799282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:35.799906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:35.800852 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:35.810671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:35.814827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:35.961167 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:35.998233 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-cbdd8cfe-4f17-4338-9c74-48463f2bed0a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1296 microversion: 2.1 time: 0.211846 Sep 30 09:48:35.999194 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 162/314] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:35 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1296 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:36.328939 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:36.336600 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:36.365147 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:36.365645 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:36.366081 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:36.380178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:36.380799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:36.381999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:37.197973 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:37.220349 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a6a0bf51-4733-4420-9a8a-d7322407ab15 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.906511 Sep 30 09:48:37.248641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 163/315] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:36 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 936 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:37.261047 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:37.263773 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:37.284723 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:37.289521 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:37.290177 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:37.291092 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:37.293624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:37.294341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:37.318745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] validate_networks() for [('19f705b3-87b0-406c-8a82-287369b1f01f', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:48:37.453951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:37.469918 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bd8b7767-8124-4428-8751-51e1b864ae43 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1316 microversion: 2.1 time: 0.216963 Sep 30 09:48:37.490130 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 164/316] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:37 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1316 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:37.497702 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-498808ae-7557-44ad-ae91-b7a5674ee556 req-62dab4ff-7d5d-41c5-8e03-3bf5b91d0ffa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35cb36cf-5501-4cf6-8a17-c988d7a0e337", "tag": "4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:37.513911 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-498808ae-7557-44ad-ae91-b7a5674ee556 req-62dab4ff-7d5d-41c5-8e03-3bf5b91d0ffa service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:37.516333 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-498808ae-7557-44ad-ae91-b7a5674ee556 req-62dab4ff-7d5d-41c5-8e03-3bf5b91d0ffa service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:37.517043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-498808ae-7557-44ad-ae91-b7a5674ee556 req-62dab4ff-7d5d-41c5-8e03-3bf5b91d0ffa service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:37.534036 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-498808ae-7557-44ad-ae91-b7a5674ee556 req-62dab4ff-7d5d-41c5-8e03-3bf5b91d0ffa service nova] Creating event network-changed:4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 for instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 on hv-cinder-maste Sep 30 09:48:37.540389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-498808ae-7557-44ad-ae91-b7a5674ee556 req-62dab4ff-7d5d-41c5-8e03-3bf5b91d0ffa service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048653 Sep 30 09:48:37.541565 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 165/317] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:37.547690 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:37.548651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:37.555812 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:37.556526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:37.557111 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:37.560028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:37.561450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:37.562235 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:37.762373 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:37.780996 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-72300526-8e94-424b-aec7-71eb43c5aac1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1296 microversion: 2.1 time: 0.235015 Sep 30 09:48:37.785198 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 166/318] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:37 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1296 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:37.797540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1939949258", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:37.884311 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] validate_networks() for None {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:48:38.890521 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:48:38.891184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:48:38.891465 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:48:38.891966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:48:38.892398 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:48:38.917137 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:38.918518 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:38.919191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:38.920252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:38.920902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:38.923996 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:38.955634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Getting quotas for project 068a17bb811448d28e4ba1907f47b8a1. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:38.971017 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:38.972736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Getting quotas for user 6ebe000d3eb147aca44db7f213183c14 and project 068a17bb811448d28e4ba1907f47b8a1. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:38.988414 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:48:38.991224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:48:39.207848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-29b6bbbb-87bc-42c4-afdc-b29ec2661e5d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.421044 Sep 30 09:48:39.218741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 167/319] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:48:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 1433 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:48:39.228176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:39.234076 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:39.251215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:39.251215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:39.251215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:39.251215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:39.251554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:39.251554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:39.463094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:48:39.463094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:48:39.463094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:48:39.463094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:48:39.463094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:48:39.471381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:39.474017 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:39.474789 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:39.475802 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:39.476503 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:39.483489 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:39.555732 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Getting quotas for project c1eb9ddbc0a84a308ceffeae5198f229. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:39.563359 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Getting quotas for user 587169e32cf64fe5b684e4b2ce0701a8 and project c1eb9ddbc0a84a308ceffeae5198f229. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:39.599698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:48:39.602193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:48:39.788041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-cc9a76fd-24bc-4695-8c57-f680a47069c1 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.025676 Sep 30 09:48:39.795555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 153/320] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:48:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 4034 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:48:39.802273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:39.818036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:39.829489 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:39.859740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5cb05502-b356-4383-9a9b-c62f7ce29923 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.636276 Sep 30 09:48:39.860572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 168/321] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:39 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 641 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:39.862188 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:39.862645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:39.893146 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:39.902127 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:39.903037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:39.906033 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:39.908428 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:39.917206 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:39.939743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:39.941692 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:39.943052 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:39.947746 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:39.951190 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:39.953647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:40.705944 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:40.726218 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a44796ad-fda3-47d7-bb74-9f1350bdfdaa tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.917429 Sep 30 09:48:40.726218 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 154/322] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:39 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 921 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:40.733375 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-06e31383-f7e3-43de-9b24-7b1fc5d0d679 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:40.734002 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-06e31383-f7e3-43de-9b24-7b1fc5d0d679 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:41.043340 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-06e31383-f7e3-43de-9b24-7b1fc5d0d679 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:41.052338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-06e31383-f7e3-43de-9b24-7b1fc5d0d679 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1215 microversion: 2.1 time: 0.333210 Sep 30 09:48:41.056111 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 155/323] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:40 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1215 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:41.062764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-18fd5313-d2f9-46c7-bbd6-fc7840d7d608 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:41.063853 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-18fd5313-d2f9-46c7-bbd6-fc7840d7d608 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:41.116696 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:41.128030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4f8248af-04fc-4afa-9a0c-6bc184467351 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 1.255193 Sep 30 09:48:41.130126 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 169/324] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:39 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 1260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:41.140997 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:41.146266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:41.159535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:41.163518 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:41.163518 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:41.164709 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:41.166772 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:41.168514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:41.207920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-18fd5313-d2f9-46c7-bbd6-fc7840d7d608 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:41.217669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-18fd5313-d2f9-46c7-bbd6-fc7840d7d608 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1214 microversion: 2.1 time: 0.160064 Sep 30 09:48:41.237428 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 156/325] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:41 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1214 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:41.746392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:41.747135 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:41.774705 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:41.775477 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:41.778851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:41.779845 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:41.782913 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:41.787225 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:41.945486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:41.961328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-250c8f4d-364d-420b-9e23-91d55248f2bf tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.828652 Sep 30 09:48:41.970959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 170/326] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:41 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 836 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:42.014756 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-8024b901-7929-47c6-9e8a-8fda0701ce3f req-fb4080cf-2f04-465a-9b6a-168f317b7b76 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "44e7298b-38f3-436b-98ed-e9157faaca5e", "tag": "411b31d5-4a9b-44b2-8ce8-22d9cb075ef9"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:42.032311 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-8024b901-7929-47c6-9e8a-8fda0701ce3f req-fb4080cf-2f04-465a-9b6a-168f317b7b76 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:42.032768 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-8024b901-7929-47c6-9e8a-8fda0701ce3f req-fb4080cf-2f04-465a-9b6a-168f317b7b76 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:42.053130 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-8024b901-7929-47c6-9e8a-8fda0701ce3f req-fb4080cf-2f04-465a-9b6a-168f317b7b76 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:42.089311 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-8024b901-7929-47c6-9e8a-8fda0701ce3f req-fb4080cf-2f04-465a-9b6a-168f317b7b76 service nova] Creating event network-changed:411b31d5-4a9b-44b2-8ce8-22d9cb075ef9 for instance 44e7298b-38f3-436b-98ed-e9157faaca5e on hv-cinder-maste Sep 30 09:48:42.095753 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-8024b901-7929-47c6-9e8a-8fda0701ce3f req-fb4080cf-2f04-465a-9b6a-168f317b7b76 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086930 Sep 30 09:48:42.096931 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 171/327] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:48:42.103152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:42.105484 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:42.114193 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:42.115072 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:42.117353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:42.117353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:42.127684 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:42.128745 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:42.197036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:42.208042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-909faa59-cb30-4b3d-b9d4-f65bd7cdca23 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.468083 Sep 30 09:48:42.211894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 157/328] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:41 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:42.228339 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:42.231231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:42.240717 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:42.241764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:42.243692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:42.243692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:42.244554 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:42.245378 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:42.284310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:42.293532 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a3614ef6-53ad-4956-8ef2-a9c999cea81b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1267 microversion: 2.1 time: 0.195130 Sep 30 09:48:42.294122 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 172/329] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:42 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1267 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:42.341470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:42.342400 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:42.365481 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:42.366229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:42.366738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:42.374814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:42.375318 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:42.375781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:43.099407 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:43.111198 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f6f55a62-09bc-44fc-b4bd-553579d26e2a tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1210 microversion: 2.1 time: 0.778117 Sep 30 09:48:43.121391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 173/330] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:42 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1210 bytes in 820 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:43.133564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:43.134309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:43.150133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:43.150959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:43.152279 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:43.154171 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:43.155012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:43.156432 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:43.218278 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:43.238513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7708240e-0981-4148-8293-355bbe2957ae tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 1.024387 Sep 30 09:48:43.239137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 158/331] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:42 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 1026 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:43.261206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:43.264758 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:43.284570 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:43.285074 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:43.285622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:43.286408 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:43.287078 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:43.287507 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:43.573562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:43.581928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-37c09ac6-f499-4d5b-9655-c14214bb5958 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.465426 Sep 30 09:48:43.595692 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 174/332] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:43 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:43.600695 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:43.601293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:43.608746 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:43.619387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:43.620255 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:43.621234 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:43.622004 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:43.622525 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:43.824668 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:43.828676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:43.833488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d9d91c5b-ee0d-4909-84bf-527680271dbd tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.577638 Sep 30 09:48:43.834719 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 159/333] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:43 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:43.863857 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1719e6c7-3da9-44f9-b37b-0a7e36acd137 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1267 microversion: 2.1 time: 0.265286 Sep 30 09:48:43.864780 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 175/334] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:43 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1267 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:44.140682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:44.141672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:44.155207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:44.160074 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:44.167026 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:44.169900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:44.169900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:44.169900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:44.271251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:44.439604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:44.439604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:44.439604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:44.439604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:44.441031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:44.441031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:44.441031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:44.441031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=100 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:44.443742 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:44.443742 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-19c6c404-54f9-4e63-bbbe-f0f486a4b6b3 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1266 microversion: 2.1 time: 0.292643 Sep 30 09:48:44.443742 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 160/335] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:44 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1266 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:44.620354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:44.621107 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:44.647541 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:44.648056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:44.652768 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:44.655297 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:44.658411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:44.659622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:44.879151 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:44.890344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-70c9d08b-9be9-40ef-881f-114ce41d24c9 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.624430 Sep 30 09:48:44.894255 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 176/336] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:44 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:44.899284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:44.907930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:44.945437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:44.945437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:44.945437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:44.945437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:44.945831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:44.945831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:45.252998 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:45.262928 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3f3fa71c-ab8a-4432-95f5-5d54baf7c7f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.649133 Sep 30 09:48:45.275755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 161/337] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:44 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:45.280691 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:45.281418 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:45.288835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:45.289419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:45.295796 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:45.296650 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:45.297263 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:45.297759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:45.352623 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:45.363014 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-60074c75-ddaa-4851-83ba-9166b44c1072 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.467397 Sep 30 09:48:45.387924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 177/338] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:44 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:45.406292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-f325d5a8-e3f0-4ea6-a2c1-f0d8cb54f7a6 req-dfea8ae5-f478-4a00-b20e-a5b3e41f0ea7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59975d37-8db0-4019-9303-88b12ef029ed", "tag": "b4db42a3-b3ea-4c10-8fa2-c3c219dd729e"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:45.448312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-f325d5a8-e3f0-4ea6-a2c1-f0d8cb54f7a6 req-dfea8ae5-f478-4a00-b20e-a5b3e41f0ea7 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:45.448875 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-f325d5a8-e3f0-4ea6-a2c1-f0d8cb54f7a6 req-dfea8ae5-f478-4a00-b20e-a5b3e41f0ea7 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:45.449212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-f325d5a8-e3f0-4ea6-a2c1-f0d8cb54f7a6 req-dfea8ae5-f478-4a00-b20e-a5b3e41f0ea7 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:45.491779 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-f325d5a8-e3f0-4ea6-a2c1-f0d8cb54f7a6 req-dfea8ae5-f478-4a00-b20e-a5b3e41f0ea7 service nova] Creating event network-changed:b4db42a3-b3ea-4c10-8fa2-c3c219dd729e for instance 59975d37-8db0-4019-9303-88b12ef029ed on hv-cinder-maste Sep 30 09:48:45.506481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:45.517414 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-f325d5a8-e3f0-4ea6-a2c1-f0d8cb54f7a6 req-dfea8ae5-f478-4a00-b20e-a5b3e41f0ea7 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.121913 Sep 30 09:48:45.518615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 178/339] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:45.519642 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4baeef09-f25d-4c40-a1d8-803b4a17204e tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1273 microversion: 2.1 time: 0.239503 Sep 30 09:48:45.520360 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 162/340] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:45 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1273 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:45.525414 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:45.527316 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:45.566663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:45.567584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:45.568127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:45.568999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:45.570324 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:45.571173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:45.643856 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:45.653266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bea160a5-1624-4350-a5cf-f1f754cbaa29 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1266 microversion: 2.1 time: 0.133331 Sep 30 09:48:45.654999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 179/341] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:45 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1266 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:45.916901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:45.917507 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:45.926190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:45.926964 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:45.927567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:45.928624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:45.929234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:45.929811 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:46.112674 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:46.123187 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4debdd09-5c3f-4f0b-8100-18cd4a24d777 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.209574 Sep 30 09:48:46.126139 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 163/342] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:45 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:46.298098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:46.298850 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:46.305886 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:46.309671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:46.310088 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:46.311698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:46.314236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:46.315967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:46.411761 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:46.412889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:46.438887 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:46.441851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:46.442344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:46.445038 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:46.447832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:46.447832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:46.786098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:46.796294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-574f872e-b850-412c-bc79-e015215b3942 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.501894 Sep 30 09:48:46.799002 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 180/343] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:46 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:46.821013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:46.836976 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:46.842146 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:46.844142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a62cc1fe-20ee-4fed-94af-74fa79210a53 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.437843 Sep 30 09:48:46.846245 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 164/344] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:46 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:46.865352 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:46.870959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:46.870959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:46.871250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:46.871341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:46.871341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:46.874079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:46.875041 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:46.888110 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:46.901345 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:46.904273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:46.905285 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:46.906203 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:46.906901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:47.500784 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:47.525011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4bedb180-9510-4caf-bebc-34402a19fd3b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1274 microversion: 2.1 time: 0.673316 Sep 30 09:48:47.538438 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 165/345] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:46 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1274 bytes in 691 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:47.544467 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:47.545519 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:47.553761 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:47.555061 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:47.555751 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:47.557454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:47.558260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:47.558942 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:47.591916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:48.097369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:48.111466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0572fa19-d6d7-4a3f-abdf-9bc6e9d8db14 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 1.303422 Sep 30 09:48:48.113233 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 181/346] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:46 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 1306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:48.122247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:48.124526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:48.138700 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:48.140130 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:48.141461 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:48.143367 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:48.144170 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:48.144940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:48.279578 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:48.296153 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1fb5209a-0b51-4ce5-b0c5-05391b139138 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.754482 Sep 30 09:48:48.303571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 166/347] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:47 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:48.320313 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:48.323910 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:48.345267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:48.345267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:48.345267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:48.352676 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:48.353176 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:48.353623 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:48.866698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:48.881346 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-56abf8d6-f92b-4338-9fed-df9da526d835 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.765054 Sep 30 09:48:48.882288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 182/348] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:48 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:48.892473 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:48.895390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:48.911183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:48.911684 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:48.914178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:48.915739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:48.918355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:48.918908 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:48.947487 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=4 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:48.984889 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=24 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:49.077667 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:49.088991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-36cd76f0-3771-449c-9e81-45c6ceec12c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.781582 Sep 30 09:48:49.103038 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 167/349] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:48 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:49.136448 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:49.137085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:49.144285 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:49.148572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:49.148572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:49.148572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:49.148572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:49.148966 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:49.191263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:49.208744 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-34b281b1-8bb9-487a-9820-4d6f6c23adb7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1272 microversion: 2.1 time: 0.324978 Sep 30 09:48:49.209633 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 183/350] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:48 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1272 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:49.330415 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:49.332113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:49.339894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:49.340518 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:49.341090 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:49.341930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:49.342621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:49.343176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:49.495559 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=6 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:49.604438 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:49.612400 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7c65af73-bfa9-404b-b490-356578de00ba tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.479942 Sep 30 09:48:49.614083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 168/351] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:49 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:49.623016 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-f8d26984-6ec3-46c9-91ce-28d625e74789 req-5b0d3aa6-562a-4648-8bb4-980e6cc195ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c655458b-a040-4717-91aa-3e156bac1764", "tag": "0f24f809-3568-48d6-bb30-cdcc4f2c72c6"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:49.632290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-f8d26984-6ec3-46c9-91ce-28d625e74789 req-5b0d3aa6-562a-4648-8bb4-980e6cc195ef service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:49.632768 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-f8d26984-6ec3-46c9-91ce-28d625e74789 req-5b0d3aa6-562a-4648-8bb4-980e6cc195ef service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:49.633192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-f8d26984-6ec3-46c9-91ce-28d625e74789 req-5b0d3aa6-562a-4648-8bb4-980e6cc195ef service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:49.644798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-f8d26984-6ec3-46c9-91ce-28d625e74789 req-5b0d3aa6-562a-4648-8bb4-980e6cc195ef service nova] Creating event network-changed:0f24f809-3568-48d6-bb30-cdcc4f2c72c6 for instance c655458b-a040-4717-91aa-3e156bac1764 on hv-cinder-maste Sep 30 09:48:49.649663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-f8d26984-6ec3-46c9-91ce-28d625e74789 req-5b0d3aa6-562a-4648-8bb4-980e6cc195ef service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035542 Sep 30 09:48:49.650476 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 169/352] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:49.656302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:49.666061 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-10e4af9f-11e6-4c44-b12d-16357069ac7e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.340436 Sep 30 09:48:49.671772 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 184/353] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:49 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:49.907592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:49.908292 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:49.915489 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:49.916417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:49.917033 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:49.918309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:49.918846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:49.919279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:50.069370 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:50.077722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-313632ca-eb8a-41cc-9e76-d3e99bfb82c5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.174859 Sep 30 09:48:50.078272 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 170/354] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:49 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:50.117671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:50.118299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:50.125046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:50.125583 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:50.126018 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:50.126735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:50.127192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:50.127617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:50.236108 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:50.236729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:50.243355 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:50.243849 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:50.244286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:50.245488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:50.245944 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:50.246372 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:50.711790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:50.722073 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c65fc8a2-8326-4404-afd9-bcd13afc826c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.608614 Sep 30 09:48:50.734952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 185/355] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:50 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:50.743460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:50.744697 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:50.750357 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:50.751124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:50.753902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:50.753902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:50.753902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:50.753902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:50.761623 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:50.772065 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-60e8201a-044a-4b25-b914-657ab12979ca tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.539930 Sep 30 09:48:50.772823 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 171/356] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:50 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:50.779879 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:50.782631 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:50.786640 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:50.787539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:50.788500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:50.789305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:50.789912 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:50.790443 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:51.037836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:51.046641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5bddafd9-75fb-4337-ae29-a6f96cb94387 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.310583 Sep 30 09:48:51.047240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 186/357] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:50 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:51.067833 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-df369313-c685-4aaf-ac60-6df10fe48edb req-91faf053-231e-45d4-b8a4-51d6f1a0104f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "44e7298b-38f3-436b-98ed-e9157faaca5e", "name": "network-vif-plugged", "status": "completed", "tag": "411b31d5-4a9b-44b2-8ce8-22d9cb075ef9"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:51.078096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-df369313-c685-4aaf-ac60-6df10fe48edb req-91faf053-231e-45d4-b8a4-51d6f1a0104f service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:51.078617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-df369313-c685-4aaf-ac60-6df10fe48edb req-91faf053-231e-45d4-b8a4-51d6f1a0104f service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:51.079080 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-df369313-c685-4aaf-ac60-6df10fe48edb req-91faf053-231e-45d4-b8a4-51d6f1a0104f service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:51.085072 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:51.091588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-df369313-c685-4aaf-ac60-6df10fe48edb req-91faf053-231e-45d4-b8a4-51d6f1a0104f service nova] Creating event network-vif-plugged:411b31d5-4a9b-44b2-8ce8-22d9cb075ef9 for instance 44e7298b-38f3-436b-98ed-e9157faaca5e on hv-cinder-maste Sep 30 09:48:51.098245 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-df369313-c685-4aaf-ac60-6df10fe48edb req-91faf053-231e-45d4-b8a4-51d6f1a0104f service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037563 Sep 30 09:48:51.099108 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 187/358] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:51.105159 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:51.105835 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:51.108973 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1b4fde07-4ce6-4e15-8c6d-e03c044a8ba0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.334182 Sep 30 09:48:51.114747 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 172/359] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:50 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:51.114899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:51.114899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:51.114899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:51.114899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:51.115247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:51.115247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:51.322965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:51.333660 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a47b51c4-61be-4eaf-bf3c-154524537e53 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.233189 Sep 30 09:48:51.334904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 188/360] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:51 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:51.750398 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:51.751261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:51.753354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=32 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:51.759075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:51.760709 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:51.782422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:51.782422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:51.782422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:51.782422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:51.799104 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:51.801272 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:51.808870 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:51.809570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:51.810010 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:51.810757 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:51.811209 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:51.811788 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:52.048787 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:52.068150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3cf8a78f-766d-476b-b4e5-fcc64b3401c0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.321285 Sep 30 09:48:52.086232 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 173/361] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:51 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:52.094434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:52.098609 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:52.102983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:52.111386 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:52.113042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:52.113955 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c083e1e5-40fd-4145-86ad-2262293caa8f tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.319867 Sep 30 09:48:52.114542 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 189/362] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:51 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:52.115214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:52.116205 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:52.118021 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:52.121658 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:52.136630 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:52.137429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:52.144640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:52.145389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:52.146232 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:52.147178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:52.147805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:52.148382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:52.745339 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:52.758207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-84ff2e7d-be58-4e9e-8d8d-84a97708a62f tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.670697 Sep 30 09:48:52.759163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 174/363] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:52 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 672 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:52.767515 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:52.769654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:52.774528 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:52.775364 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:52.776693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:52.777562 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:52.779007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:52.780092 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:52.781229 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:52.783679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1fc7ab2f-88da-46b0-8641-ef7a74b57216 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.651323 Sep 30 09:48:52.784686 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 190/364] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:52 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:53.095024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:53.095732 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:53.112581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.115470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.117232 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:53.118101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.118614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.119068 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:53.237798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:53.247429 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b1eb4806-1ee1-4fd9-8ca5-dc50cabebd74 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.484011 Sep 30 09:48:53.252046 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 175/365] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:52 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:53.256965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:53.272623 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:53.274911 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.275439 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.275934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:53.276766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.277283 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.277879 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:53.591272 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:53.596611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:53.603986 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8da7cb10-844a-4a8f-aab6-eea9d331b826 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1372 microversion: 2.1 time: 0.514117 Sep 30 09:48:53.605014 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-32834249-282f-444f-bd29-cd7baf629cab tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.351730 Sep 30 09:48:53.621187 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 176/366] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:53 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:53.621187 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-fcc30f52-0a3b-4d96-a2eb-1393217bce54 req-49d44c87-01f4-4326-a13c-475ce4370992 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d73628ce-2a02-4ff6-97e2-3f2f9cb0087c", "tag": "ce8d050b-ac4d-44ae-a002-9cb0ed2f51fd"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:53.623238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 191/367] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:53 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1372 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:53.647610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-fcc30f52-0a3b-4d96-a2eb-1393217bce54 req-49d44c87-01f4-4326-a13c-475ce4370992 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.647610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-fcc30f52-0a3b-4d96-a2eb-1393217bce54 req-49d44c87-01f4-4326-a13c-475ce4370992 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.648086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-fcc30f52-0a3b-4d96-a2eb-1393217bce54 req-49d44c87-01f4-4326-a13c-475ce4370992 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:53.669471 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-fcc30f52-0a3b-4d96-a2eb-1393217bce54 req-49d44c87-01f4-4326-a13c-475ce4370992 service nova] Creating event network-changed:ce8d050b-ac4d-44ae-a002-9cb0ed2f51fd for instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c on hv-cinder-maste Sep 30 09:48:53.683036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-fcc30f52-0a3b-4d96-a2eb-1393217bce54 req-49d44c87-01f4-4326-a13c-475ce4370992 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065384 Sep 30 09:48:53.683036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 177/368] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:53.781630 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:53.782246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:53.789089 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.789672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.790508 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:53.793554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.793554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.793554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:53.810394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:53.811553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:53.820540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.821030 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.821574 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:53.822890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:53.823552 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:53.823985 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:54.034752 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:54.043599 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2f837594-25a4-4897-827c-19fb3798c313 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.265658 Sep 30 09:48:54.044520 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 192/369] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:53 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:54.125950 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:54.137354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-72b4a31b-341d-4540-9be7-471f76100ead tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.333402 Sep 30 09:48:54.156942 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 178/370] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:53 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:54.275263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:54.276564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:54.284917 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:54.285733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:54.286348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:54.287312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:54.288079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:54.288663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:54.630455 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:54.631122 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:54.648086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:54.648086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:54.648086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:54.648710 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:54.649163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:54.649586 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:54.678387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:54.698911 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-96b9ae26-76dc-42eb-b4e0-aa609f4f7ec0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.427465 Sep 30 09:48:54.722172 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 193/371] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:54 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:54.728246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:54.729882 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:54.750852 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:54.752721 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:54.753441 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:54.754372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:54.755505 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:54.756104 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:54.792058 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=100 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:55.168583 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:55.183034 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-864f0673-4588-4631-a047-89006f3c1f70 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1565 microversion: 2.1 time: 0.462491 Sep 30 09:48:55.183988 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 194/372] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:54 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1565 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:55.237851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:55.247397 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4e80f5ec-c687-49cb-8797-2e0665a903a0 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.622446 Sep 30 09:48:55.249058 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 179/373] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:54 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 625 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:55.254501 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:55.255674 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:55.274838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.275916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.276576 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.277389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.278256 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.278256 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.473769 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d207822a-d327-44fa-8301-e508ffcbded1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Action: 'create', calling method: .version_select at 0x7f72c0e28790>, body: {"keypair": {"name": "tempest-TestShelveInstance-1312266241", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJkEG3ryS4BR8Q9ADsP8zEQgTRnbY04QE/gZqUIvFpso8QK6YPkYUUrFUarJJ5yQnlXQR7ZZKJN4q46xZyRUOFmlBe2K8S6OfY/DxtEt0q8UmZfD63JJKAh/ffTENkwgug=="}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:55.490568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-d207822a-d327-44fa-8301-e508ffcbded1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Getting quotas for project 1330031e214045bf9d45c73da1445255. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:48:55.501570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-d207822a-d327-44fa-8301-e508ffcbded1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Getting quotas for user 20bee9f38eb14541a393037eaa389a44 and project 1330031e214045bf9d45c73da1445255. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:48:55.569425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:55.578766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-91e67621-0c4b-4d7e-89ca-c47084a7937b tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.328312 Sep 30 09:48:55.579346 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 180/374] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:55 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:55.598717 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:55.598717 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:55.603831 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.604323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.604907 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.605625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.606077 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.606917 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.630453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d207822a-d327-44fa-8301-e508ffcbded1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.445335 Sep 30 09:48:55.638826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 195/375] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:48:55 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:55.659433 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:55.659433 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:55.671688 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.672319 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.672854 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.673766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.674291 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.675010 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.802977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-80c80e3a-1669-43fd-adea-0e204aa2060c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.168136 Sep 30 09:48:55.804510 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 196/376] 10.222.0.39 () {58 vars in 1275 bytes} [Fri Sep 30 09:48:55 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-interface => generated 284 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:55.812163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-53b72418-8cce-424b-8edf-4bf6c7a429fe req-b36af4a6-ef44-431d-800a-8df69506f734 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35cb36cf-5501-4cf6-8a17-c988d7a0e337", "name": "network-vif-plugged", "status": "completed", "tag": "4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:55.836854 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-53b72418-8cce-424b-8edf-4bf6c7a429fe req-b36af4a6-ef44-431d-800a-8df69506f734 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.838533 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-53b72418-8cce-424b-8edf-4bf6c7a429fe req-b36af4a6-ef44-431d-800a-8df69506f734 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.839571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-53b72418-8cce-424b-8edf-4bf6c7a429fe req-b36af4a6-ef44-431d-800a-8df69506f734 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.854420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-53b72418-8cce-424b-8edf-4bf6c7a429fe req-b36af4a6-ef44-431d-800a-8df69506f734 service nova] Creating event network-vif-plugged:4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 for instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 on hv-cinder-maste Sep 30 09:48:55.863995 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-53b72418-8cce-424b-8edf-4bf6c7a429fe req-b36af4a6-ef44-431d-800a-8df69506f734 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057759 Sep 30 09:48:55.865877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 197/377] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:48:55.874465 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:55.876441 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:55.887457 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.888640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.893288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.893288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:55.893288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:55.893288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:55.933693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:55.945716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8fc4e3df-23ed-4fd6-a502-06f9e1260b19 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.361807 Sep 30 09:48:55.946387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 181/378] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:55 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:56.233756 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:56.241856 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2d5596d8-8904-4ad0-99af-8abf70a3e6e6 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.373989 Sep 30 09:48:56.242421 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 198/379] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:55 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:56.272977 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:56.273643 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:56.280450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:56.280982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:56.281416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:56.282120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:56.282816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:56.283482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:56.606050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:56.606843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:56.625012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:56.626666 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:56.627947 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:56.630423 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:56.631850 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:56.633486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:57.093620 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:57.108433 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f9116b84-c1d9-40eb-96c4-0332e803f84b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.839194 Sep 30 09:48:57.123562 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 182/380] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:56 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 855 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:57.138458 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:57.146030 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:57.147799 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:57.148626 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:57.149248 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:57.150204 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:57.150938 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:57.151504 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:57.397310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:57.406025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-cc7c6e91-6341-467c-982b-97cd82a8e60c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.806207 Sep 30 09:48:57.406623 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 199/381] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:56 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 808 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:57.412425 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:57.413098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:57.419890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:57.425911 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:57.425911 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:57.426564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:57.427924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:57.428429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:57.660652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:57.677655 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a57a9379-867c-4915-88ea-58df9b33316a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.551879 Sep 30 09:48:57.679487 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 183/382] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:57 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:57.713226 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:57.723875 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a2cf606a-9fa0-4348-a3f0-facd29fb537e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.316230 Sep 30 09:48:57.725069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 200/383] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:57 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:58.136505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:58.137348 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:58.147148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.147515 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.147953 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.148682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.151772 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.152271 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.218239 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:58.331146 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-81c1f52f-ce3d-4e80-a2b6-c46737fbc0d2 req-e28c3b6c-7473-4cca-a676-ec7c95d839e0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "44e7298b-38f3-436b-98ed-e9157faaca5e", "tag": "411b31d5-4a9b-44b2-8ce8-22d9cb075ef9"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:58.362353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-81c1f52f-ce3d-4e80-a2b6-c46737fbc0d2 req-e28c3b6c-7473-4cca-a676-ec7c95d839e0 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.362583 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-81c1f52f-ce3d-4e80-a2b6-c46737fbc0d2 req-e28c3b6c-7473-4cca-a676-ec7c95d839e0 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.363212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-81c1f52f-ce3d-4e80-a2b6-c46737fbc0d2 req-e28c3b6c-7473-4cca-a676-ec7c95d839e0 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.367912 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:58.380793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4962d5cb-7ed6-44c0-8fa1-d82b54455b6b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.248309 Sep 30 09:48:58.391118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-81c1f52f-ce3d-4e80-a2b6-c46737fbc0d2 req-e28c3b6c-7473-4cca-a676-ec7c95d839e0 service nova] Creating event network-changed:411b31d5-4a9b-44b2-8ce8-22d9cb075ef9 for instance 44e7298b-38f3-436b-98ed-e9157faaca5e on hv-cinder-maste Sep 30 09:48:58.391693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 184/384] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:58 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:58.398364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-81c1f52f-ce3d-4e80-a2b6-c46737fbc0d2 req-e28c3b6c-7473-4cca-a676-ec7c95d839e0 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067227 Sep 30 09:48:58.401028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 201/385] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:48:58.448817 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:58.450339 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:58.460503 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:48:58.466485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.467097 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.470132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.470132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.470132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.470132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.632082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:58.645320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9a8e630a-909e-40c9-9f78-c4c146ba241c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.215579 Sep 30 09:48:58.645320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 185/386] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:58 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:58.709295 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:58.710005 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:58.721721 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.722351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.722735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.723348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.723794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.724207 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.750661 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:58.751563 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:58.760450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.761317 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.761977 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.762917 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:58.763720 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:58.764366 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:58.987140 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:58.998716 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-21a0a468-dfb5-4da8-b09b-40afca934284 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.301865 Sep 30 09:48:59.011891 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 202/387] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:58 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:59.061073 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:59.070575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7eb6eeb4-1c90-47a0-a363-6324df504f40 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1359 microversion: 2.1 time: 0.323580 Sep 30 09:48:59.072488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 186/388] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:58 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1359 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:48:59.405723 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:59.406332 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:59.412558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:59.412968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:59.413408 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:59.414109 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:59.414566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:59.418722 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:59.671583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:48:59.671583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:48:59.681654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:59.686559 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:59.687041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:59.687802 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:59.688264 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:59.688692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:48:59.698453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:48:59.710434 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-84cc6c14-8520-48bc-a8e9-5acaa073c3c6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.306639 Sep 30 09:48:59.711048 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 203/389] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:59 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:48:59.977865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-df9d033e-8060-45f4-889d-0dab5560a70f req-2cee35fb-9577-43eb-bebb-ffddba6d8f85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59975d37-8db0-4019-9303-88b12ef029ed", "name": "network-vif-plugged", "status": "completed", "tag": "b4db42a3-b3ea-4c10-8fa2-c3c219dd729e"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:48:59.994706 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-df9d033e-8060-45f4-889d-0dab5560a70f req-2cee35fb-9577-43eb-bebb-ffddba6d8f85 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:48:59.995186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-df9d033e-8060-45f4-889d-0dab5560a70f req-2cee35fb-9577-43eb-bebb-ffddba6d8f85 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:48:59.995610 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-df9d033e-8060-45f4-889d-0dab5560a70f req-2cee35fb-9577-43eb-bebb-ffddba6d8f85 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.010331 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-df9d033e-8060-45f4-889d-0dab5560a70f req-2cee35fb-9577-43eb-bebb-ffddba6d8f85 service nova] Creating event network-vif-plugged:b4db42a3-b3ea-4c10-8fa2-c3c219dd729e for instance 59975d37-8db0-4019-9303-88b12ef029ed on hv-cinder-maste Sep 30 09:49:00.015837 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-df9d033e-8060-45f4-889d-0dab5560a70f req-2cee35fb-9577-43eb-bebb-ffddba6d8f85 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042993 Sep 30 09:49:00.016854 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 204/390] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:48:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:00.026307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:00.027430 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:00.036830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.036830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.036830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.036830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.037231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.037487 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.044139 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:00.051883 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9870d528-d6db-45fa-a311-28f3ac72fa21 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.388527 Sep 30 09:49:00.055045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 187/391] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:48:59 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:00.093217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:00.094700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:00.101142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.101639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.102077 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.102822 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.103804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.104436 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.257882 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:00.266414 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f45becb5-3d3c-40bf-b2c7-05e62e4b3e0b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.245620 Sep 30 09:49:00.267199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 205/392] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:00 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:00.285001 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:00.307241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d0208188-4773-4cb1-bf26-ed553fea1301 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1548 microversion: 2.1 time: 0.204009 Sep 30 09:49:00.307241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 188/393] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:00 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1548 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:00.322373 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:00.324205 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:00.329490 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.329987 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.330458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.332124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.332947 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.333580 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.734279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:00.734924 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:00.742084 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.742500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.742959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.743820 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.744139 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.744576 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.777676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:00.818737 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e62d4d1d-8d4f-4107-aeac-a9e8a9f4f7f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1548 microversion: 2.1 time: 0.503976 Sep 30 09:49:00.819327 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 206/394] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:00 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1548 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:00.842980 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:00.848317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:00.849258 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Searching by: {'deleted': False, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1'} {{(pid=127456) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 30 09:49:00.856563 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.857135 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.857563 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.858930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:00.859391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:00.859813 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:00.961973 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:00.967480 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.multi_cell_list [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=127456) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 30 09:49:00.970231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b23b9b82-a449-46e3-ac4b-eaf9d11d8d19 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.240037 Sep 30 09:49:00.970987 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 189/395] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:00 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:01.059448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5aa0e863-dd79-4457-ba25-7bc03d0a7498 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.221461 Sep 30 09:49:01.060033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 207/396] 10.222.0.39 () {58 vars in 1125 bytes} [Fri Sep 30 09:49:00 2022] GET /compute/v2.1/servers => generated 343 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:49:01.071547 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:01.076298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:01.078730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:01.078880 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:01.079289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:01.079418 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:01.079718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:01.080411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:01.080861 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:01.081282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:01.081451 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:01.086048 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:01.086533 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:01.087009 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:01.088451 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:01.088909 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:01.089335 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:01.277013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:01.281415 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:01.288014 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-44440f25-2319-4185-8739-f2e20cb321ac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1548 microversion: 2.1 time: 0.220256 Sep 30 09:49:01.288588 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 190/397] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:01 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1548 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:01.289755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-32051c64-3698-486a-a877-3f64f857788d tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.214953 Sep 30 09:49:01.290721 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 208/398] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:01 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:01.293793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:01.294564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:01.306495 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:01.307007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:01.307716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:01.308212 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:01.310982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:01.311427 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:01.444650 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:01.453012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-bb1bc70f-34be-4a26-a8fb-c48b4765bccf tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.163264 Sep 30 09:49:01.453576 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 191/399] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:01 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:01.995882 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:01.996959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:02.003531 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:02.004136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:02.004587 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:02.005327 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:02.005760 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:02.006189 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:02.035477 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-181c263d-525f-409c-a7ed-4ca2b7b0a821 req-6c93afbb-700e-4c4b-ac00-3c61c5ea2043 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c655458b-a040-4717-91aa-3e156bac1764", "name": "network-vif-plugged", "status": "completed", "tag": "0f24f809-3568-48d6-bb30-cdcc4f2c72c6"}, {"server_uuid": "d73628ce-2a02-4ff6-97e2-3f2f9cb0087c", "name": "network-vif-plugged", "status": "completed", "tag": "ce8d050b-ac4d-44ae-a002-9cb0ed2f51fd"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:02.046814 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-181c263d-525f-409c-a7ed-4ca2b7b0a821 req-6c93afbb-700e-4c4b-ac00-3c61c5ea2043 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:02.047300 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-181c263d-525f-409c-a7ed-4ca2b7b0a821 req-6c93afbb-700e-4c4b-ac00-3c61c5ea2043 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:02.047809 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-181c263d-525f-409c-a7ed-4ca2b7b0a821 req-6c93afbb-700e-4c4b-ac00-3c61c5ea2043 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:02.061270 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-181c263d-525f-409c-a7ed-4ca2b7b0a821 req-6c93afbb-700e-4c4b-ac00-3c61c5ea2043 service nova] Creating event network-vif-plugged:0f24f809-3568-48d6-bb30-cdcc4f2c72c6 for instance c655458b-a040-4717-91aa-3e156bac1764 on hv-cinder-maste Sep 30 09:49:02.061922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-181c263d-525f-409c-a7ed-4ca2b7b0a821 req-6c93afbb-700e-4c4b-ac00-3c61c5ea2043 service nova] Creating event network-vif-plugged:ce8d050b-ac4d-44ae-a002-9cb0ed2f51fd for instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c on hv-cinder-maste Sep 30 09:49:02.068856 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-181c263d-525f-409c-a7ed-4ca2b7b0a821 req-6c93afbb-700e-4c4b-ac00-3c61c5ea2043 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.039974 Sep 30 09:49:02.069741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 192/400] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:02 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:49:02.151306 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:02.159075 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f5256f4c-35a5-4b7c-bc82-37d3fc66ae8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.167548 Sep 30 09:49:02.159641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 209/401] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:01 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:02.313337 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:02.314118 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:02.320830 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:02.321315 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:02.322258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:02.323040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:02.323488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:02.323969 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:02.451558 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:02.459053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2db1cfe3-f774-4052-a552-0f8fd2e46d77 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.150015 Sep 30 09:49:02.459616 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 193/402] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:02 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:02.475458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:02.476100 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:02.482018 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:02.482504 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:02.482973 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:02.483658 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:02.484108 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:02.484528 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:02.621622 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:02.629238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-69a3e58b-0787-470d-986d-f0b203a7cba5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1363 microversion: 2.1 time: 0.157609 Sep 30 09:49:02.629832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 210/403] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:02 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:03.185645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:03.186241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:03.197730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.198539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.199766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.201649 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.202144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.202633 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.275771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9392ad7b-d061-4ce1-b5ac-59db92be7d11"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:03.278994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:03.286749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.291869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.292613 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.293568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.294893 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.295555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.446852 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:03.455275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-bd648b2b-0557-4349-99d2-d26fd93356fc tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.273891 Sep 30 09:49:03.455862 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 194/404] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:03 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:03.481547 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:03.489399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:03.489399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.489399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.489399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.489865 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.490327 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.490789 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.622738 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:03.630609 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-65a32bf5-5924-4a9f-ad47-101e9341481c tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.153408 Sep 30 09:49:03.631168 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 195/405] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:03 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:03.652358 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:03.652955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:03.658720 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.659197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.659620 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.660290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.660735 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.661146 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.794858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:03.802467 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-39a4e8bb-0dde-4f9a-8ce5-eef4b75783f7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1553 microversion: 2.1 time: 0.154289 Sep 30 09:49:03.803060 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 196/406] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:03 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1553 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:03.824487 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:03.824982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:03.831486 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.831972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.832406 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.833098 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:03.833551 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:03.833973 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:03.911847 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7f5ff1ee-9350-483c-a08c-85ed88f761e5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.091603 Sep 30 09:49:03.912447 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 197/407] 10.222.0.39 () {58 vars in 1275 bytes} [Fri Sep 30 09:49:03 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed/os-interface => generated 284 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:49:04.479798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:04.480427 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:04.486620 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:04.487542 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:04.488235 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:04.488692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:04.489114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:04.745583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:04.753582 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c47755b1-53f1-4683-9322-4616128cb7a6 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1314 microversion: 2.1 time: 0.278662 Sep 30 09:49:04.754153 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 198/408] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:04 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1314 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:04.758733 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:04.759332 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:04.765056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:04.765533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:04.765964 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:04.766746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:04.767200 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:04.767622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:04.995890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:05.003615 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-aca2e287-c8f1-4632-bf8c-a5cafc70fb6a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1315 microversion: 2.1 time: 0.248416 Sep 30 09:49:05.004405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 199/409] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:04 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1315 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:05.097673 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-7bd39e8d-c4be-447f-af4b-24cdc322f731 req-1b8523f0-fb16-41e9-a4c7-9bb12a8965b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59975d37-8db0-4019-9303-88b12ef029ed", "tag": "b4db42a3-b3ea-4c10-8fa2-c3c219dd729e"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:05.107357 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-7bd39e8d-c4be-447f-af4b-24cdc322f731 req-1b8523f0-fb16-41e9-a4c7-9bb12a8965b1 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:05.107877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-7bd39e8d-c4be-447f-af4b-24cdc322f731 req-1b8523f0-fb16-41e9-a4c7-9bb12a8965b1 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:05.108299 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-7bd39e8d-c4be-447f-af4b-24cdc322f731 req-1b8523f0-fb16-41e9-a4c7-9bb12a8965b1 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:05.124893 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-7bd39e8d-c4be-447f-af4b-24cdc322f731 req-1b8523f0-fb16-41e9-a4c7-9bb12a8965b1 service nova] Creating event network-changed:b4db42a3-b3ea-4c10-8fa2-c3c219dd729e for instance 59975d37-8db0-4019-9303-88b12ef029ed on hv-cinder-maste Sep 30 09:49:05.130084 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-7bd39e8d-c4be-447f-af4b-24cdc322f731 req-1b8523f0-fb16-41e9-a4c7-9bb12a8965b1 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035768 Sep 30 09:49:05.130979 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 200/410] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:49:05.507808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=4 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:05.778164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:05.779005 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:05.785806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:05.786368 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:05.786895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:05.787600 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:05.788052 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:05.788472 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:05.929220 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:05.938962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b21d34db-19b7-47da-8cdf-01a82f4fec20 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1497 microversion: 2.1 time: 0.165779 Sep 30 09:49:05.942618 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 201/411] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:05 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:06.030470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:06.031337 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:06.038035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:06.038709 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:06.039271 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:06.040110 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:06.040689 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:06.041206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:06.190528 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:06.193464 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:06.206729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3b60fbb5-2390-4ed2-b659-6e172361d6dd tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1465 microversion: 2.1 time: 0.183186 Sep 30 09:49:06.208301 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 202/412] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:06 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1465 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:06.519763 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:06.520308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:06.526637 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:06.527109 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:06.527537 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:06.528398 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:06.528843 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:06.529247 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:06.557273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:49:06.782448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d5e9cf86-01a5-4a73-92cc-1af60b4a5053 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "POST /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.511488 Sep 30 09:49:06.783980 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 211/413] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:49:03 2022] POST /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments => generated 194 bytes in 3514 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:07.016475 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-83a474e5-55e5-4e99-a2f4-1b96b79a2bef tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "DELETE /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 204 len: 0 microversion: 2.1 time: 0.500468 Sep 30 09:49:07.017217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 203/414] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:49:06 2022] DELETE /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 30 09:49:07.032385 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:07.033086 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:07.040152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:07.041140 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:07.041631 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:07.042678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:07.043179 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:07.047548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:07.158307 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:07.158914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:07.164739 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:07.165640 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:07.166822 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:07.167700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:07.168316 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:07.168910 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:07.202280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:49:07.290007 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:07.294182 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:07.298871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1c96d907-d69d-4883-a294-4c780cdd8ef1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1471 microversion: 2.1 time: 0.270774 Sep 30 09:49:07.299436 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 212/415] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:07 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1471 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:07.459279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ab8832f5-0939-4814-9920-214bb0ca7c7d tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "DELETE /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 204 len: 0 microversion: 2.1 time: 0.304796 Sep 30 09:49:07.459791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 204/416] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:49:07 2022] DELETE /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 30 09:49:07.468905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:07.469521 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:07.475845 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:07.476332 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:07.476770 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:07.477461 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:07.477920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:07.479061 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:07.608588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:07.616581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-740ccd66-f8b2-4a0b-8c75-9ca3df3be893 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1503 microversion: 2.1 time: 0.151668 Sep 30 09:49:07.617293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 213/417] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:07 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1503 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:08.322362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:08.323161 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:08.329949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:08.330473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:08.330946 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:08.331646 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:08.332146 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:08.332571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:08.460454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:08.468095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8f7b5d8e-2afe-437a-9bd0-740bcfa247e0 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1471 microversion: 2.1 time: 0.150029 Sep 30 09:49:08.468672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 205/418] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:08 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1471 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:08.640345 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:08.640869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:08.647216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:08.647698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:08.648129 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:08.648817 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:08.649272 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:08.649693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:08.823438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:08.831095 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9ac2fe9c-1799-41ca-83b8-043380022050 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1503 microversion: 2.1 time: 0.194743 Sep 30 09:49:08.831656 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 214/419] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:08 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1503 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:09.493124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:09.494213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:09.501089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:09.501571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:09.502019 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:09.502733 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:09.503341 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:09.503760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:09.636873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:09.644494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c0903d3d-1f50-4b87-a678-0dc56d9969a1 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1471 microversion: 2.1 time: 0.155714 Sep 30 09:49:09.645055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 206/420] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:09 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1471 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:09.855133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:09.855666 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:09.861958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:09.862434 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:09.862903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:09.863585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:09.864034 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:09.864453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:10.001829 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:10.009711 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f1d79cf3-6726-445d-b8b0-2f778542e177 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1503 microversion: 2.1 time: 0.159151 Sep 30 09:49:10.014437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 215/421] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:09 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1503 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:10.667360 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:10.668046 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:10.674394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:10.674908 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:10.675342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:10.676642 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:10.677097 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:10.677517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:10.808901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:10.816571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3e7d6789-0bfc-44ef-ad62-05fa63ddefaa tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1471 microversion: 2.1 time: 0.153088 Sep 30 09:49:10.817138 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 207/422] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:10 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1471 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:11.040318 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:11.041935 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:11.055959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:11.058659 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:11.059139 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:11.059789 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:11.060714 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:11.061375 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:11.062002 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:11.376023 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:11.385530 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d67fd0dd-d32d-4779-981a-7d0be99251e9 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1503 microversion: 2.1 time: 0.356170 Sep 30 09:49:11.386340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 216/423] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:11 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1503 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:11.669320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-4ba5897b-f034-4517-8f85-40439673f20d req-75a47b98-bded-4e80-882d-88251d61b462 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c655458b-a040-4717-91aa-3e156bac1764", "name": "network-vif-deleted", "tag": "0f24f809-3568-48d6-bb30-cdcc4f2c72c6"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:11.685140 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-4ba5897b-f034-4517-8f85-40439673f20d req-75a47b98-bded-4e80-882d-88251d61b462 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:11.685652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-4ba5897b-f034-4517-8f85-40439673f20d req-75a47b98-bded-4e80-882d-88251d61b462 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:11.686040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-4ba5897b-f034-4517-8f85-40439673f20d req-75a47b98-bded-4e80-882d-88251d61b462 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:11.703008 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-4ba5897b-f034-4517-8f85-40439673f20d req-75a47b98-bded-4e80-882d-88251d61b462 service nova] Creating event network-vif-deleted:0f24f809-3568-48d6-bb30-cdcc4f2c72c6 for instance c655458b-a040-4717-91aa-3e156bac1764 on hv-cinder-maste Sep 30 09:49:11.708030 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-4ba5897b-f034-4517-8f85-40439673f20d req-75a47b98-bded-4e80-882d-88251d61b462 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055375 Sep 30 09:49:11.709037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 208/424] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:11.725941 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-5fa5d48d-1e28-48b2-892d-e0b80f4b4684 req-d3d33446-f821-4551-87e8-bda52c44160f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d73628ce-2a02-4ff6-97e2-3f2f9cb0087c", "name": "network-vif-deleted", "tag": "ce8d050b-ac4d-44ae-a002-9cb0ed2f51fd"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:11.737056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-5fa5d48d-1e28-48b2-892d-e0b80f4b4684 req-d3d33446-f821-4551-87e8-bda52c44160f service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:11.737623 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-5fa5d48d-1e28-48b2-892d-e0b80f4b4684 req-d3d33446-f821-4551-87e8-bda52c44160f service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:11.738107 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-5fa5d48d-1e28-48b2-892d-e0b80f4b4684 req-d3d33446-f821-4551-87e8-bda52c44160f service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:11.755587 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-5fa5d48d-1e28-48b2-892d-e0b80f4b4684 req-d3d33446-f821-4551-87e8-bda52c44160f service nova] Creating event network-vif-deleted:ce8d050b-ac4d-44ae-a002-9cb0ed2f51fd for instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c on hv-cinder-maste Sep 30 09:49:11.761432 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-5fa5d48d-1e28-48b2-892d-e0b80f4b4684 req-d3d33446-f821-4551-87e8-bda52c44160f service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039028 Sep 30 09:49:11.761965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 217/425] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:11.840791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:11.841408 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:11.847580 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:11.848057 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:11.848486 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:11.849173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:11.849633 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:11.850042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:11.914931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:11.923739 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-026d880d-311f-4134-bdbc-9ef296a36c63 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1300 microversion: 2.1 time: 0.087797 Sep 30 09:49:11.924366 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 209/426] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:11 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1300 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:12.408812 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:12.409502 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:12.416084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:12.416558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:12.416986 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:12.418017 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:12.418480 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:12.418927 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:12.451985 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:12.452458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:12.452952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:12.453449 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lazy-loading 'fault' on Instance uuid d73628ce-2a02-4ff6-97e2-3f2f9cb0087c {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:49:12.496789 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:12.506113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2a08b198-119d-40a6-b117-bc1c4070ba8b tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1304 microversion: 2.1 time: 0.101340 Sep 30 09:49:12.507090 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 218/427] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:12 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1304 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:12.947287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:12.947827 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:12.954233 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:12.954745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:12.955181 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:12.955899 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:12.956351 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:12.956772 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:12.987934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:12.988416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:12.988850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:12.989290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lazy-loading 'fault' on Instance uuid c655458b-a040-4717-91aa-3e156bac1764 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:49:13.033115 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ebe000d3eb147aca44db7f213183c14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '068a17bb811448d28e4ba1907f47b8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:13.042406 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1169691f-3913-4007-bce9-96c4da73ab31 tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 200 len: 1305 microversion: 2.1 time: 0.099046 Sep 30 09:49:13.043027 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 210/428] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:12 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 1305 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:13.532515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:13.532515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:13.538172 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:13.539103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:13.539524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:13.540189 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:13.540738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:13.541051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:13.576952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:13.577434 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:13.577866 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:13.578300 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lazy-loading 'fault' on Instance uuid d73628ce-2a02-4ff6-97e2-3f2f9cb0087c {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:49:13.618171 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '587169e32cf64fe5b684e4b2ce0701a8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1eb9ddbc0a84a308ceffeae5198f229', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:13.625667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-10301393-f921-43cc-8b81-70325b9320dd tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 200 len: 1304 microversion: 2.1 time: 0.099004 Sep 30 09:49:13.626241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 219/429] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:13 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 1304 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:14.066462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:14.067101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] [instance: c655458b-a040-4717-91aa-3e156bac1764] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:14.074088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:14.074615 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:14.075058 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:14.075856 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:14.076224 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:14.076645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:14.089865 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] HTTP exception thrown: Instance c655458b-a040-4717-91aa-3e156bac1764 could not be found. Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Request method failure captured: Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 HTTP/1.1 Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrtxsMbG_ZsdX2T_oafgLGHYr_j6yr7X8jnKm0sxStgKukIrDVY5K6IIiJPA1e-GuCIQcYTPKTOvg9WNEtwkMAAzcRZdyhtCQspmXAGRmSP8ZfGo5trrul1lus0XwExVq7NdjuTCfq1Ype49orTJ5N5H509LrG3FErsizGAT8F-9M40 Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 068a17bb811448d28e4ba1907f47b8a1 Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-VolumesNegativeTest-1429228178 Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:49:14.091575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/068a17bb811448d28e4ba1907f47b8a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/068a17bb811448d28e4ba1907f47b8a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/068a17bb811448d28e4ba1907f47b8a1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_068a17bb811448d28e4ba1907f47b8a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-VolumesNegativeTest-1429228178 Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 068a17bb811448d28e4ba1907f47b8a1 Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-VolumesNegativeTest-1429228178 Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-VolumesNegativeTest-1429228178-project Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 6ebe000d3eb147aca44db7f213183c14 Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-VolumesNegativeTest-1429228178-project Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance c655458b-a040-4717-91aa-3e156bac1764 could not be found. Sep 30 09:49:14.093165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': 'c655458b-a040-4717-91aa-3e156bac1764'} Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c655458b-a040-4717-91aa-3e156bac1764 could not be found. Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:49:14.093976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c655458b-a040-4717-91aa-3e156bac1764 could not be found. Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c655458b-a040-4717-91aa-3e156bac1764 could not be found. Sep 30 09:49:14.095613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.097519 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] Returning 404 to user: Instance c655458b-a040-4717-91aa-3e156bac1764 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:49:14.097519 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-542872a9-5ab0-4d0f-8f0e-089fd130562a tempest-VolumesNegativeTest-1429228178 tempest-VolumesNegativeTest-1429228178-project] 10.222.0.39 "GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764" status: 404 len: 111 microversion: 2.1 time: 0.030377 Sep 30 09:49:14.097519 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 211/430] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:14 2022] GET /compute/v2.1/servers/c655458b-a040-4717-91aa-3e156bac1764 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:49:14.649048 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:14.649731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] [instance: d73628ce-2a02-4ff6-97e2-3f2f9cb0087c] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:14.656635 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:14.657113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:14.657542 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:14.658226 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:14.658700 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:14.661323 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:14.672671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] HTTP exception thrown: Instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c could not be found. Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Request method failure captured: Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c HTTP/1.1 Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrtKwaffcg461ol3LZNOL1Yy5_5CkyV3Y_EfWjrrUVQYdeYaK3kNuLZ0_ojEdr86yPbtav26zSgCHXWPEXcJLBKFISfTCeemF7xQPK1GMOickHtS25WzWVMeTFpKuqRmN1J3MV3QbIPSXgL1kos5fSqeoMbC-ZnEvnei6T9RU8SCN78 Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: c1eb9ddbc0a84a308ceffeae5198f229 Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-VolumesActionsTest-1876305362 Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:49:14.674277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/c1eb9ddbc0a84a308ceffeae5198f229"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/c1eb9ddbc0a84a308ceffeae5198f229"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/c1eb9ddbc0a84a308ceffeae5198f229"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_c1eb9ddbc0a84a308ceffeae5198f229"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-VolumesActionsTest-1876305362 Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: c1eb9ddbc0a84a308ceffeae5198f229 Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-VolumesActionsTest-1876305362 Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-VolumesActionsTest-1876305362-project Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 587169e32cf64fe5b684e4b2ce0701a8 Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-VolumesActionsTest-1876305362-project Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c could not be found. Sep 30 09:49:14.675830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': 'd73628ce-2a02-4ff6-97e2-3f2f9cb0087c'} Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c could not be found. Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:49:14.676802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c could not be found. Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c could not be found. Sep 30 09:49:14.678225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:49:14.679562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] Returning 404 to user: Instance d73628ce-2a02-4ff6-97e2-3f2f9cb0087c could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:49:14.679562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-06fe5b29-121d-4b5e-87f0-9edd0333cad7 tempest-VolumesActionsTest-1876305362 tempest-VolumesActionsTest-1876305362-project] 10.222.0.39 "GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c" status: 404 len: 111 microversion: 2.1 time: 0.030406 Sep 30 09:49:14.679562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 220/431] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:14 2022] GET /compute/v2.1/servers/d73628ce-2a02-4ff6-97e2-3f2f9cb0087c => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:49:20.916917 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:20.918201 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:20.924938 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:20.925423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:20.925855 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:20.926544 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:20.927030 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:20.927452 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:21.057348 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:21.065600 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-39641603-8980-4892-94cf-c9a9bf9399f4 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1594 microversion: 2.1 time: 0.152538 Sep 30 09:49:21.066195 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 212/432] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:20 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1594 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:22.111564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1496875305", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1312266241", "security_groups": [{"name": "tempest-secgroup-smoke-783740169"}], "networks": [{"uuid": "66e518e7-6dfd-46cc-b9d0-1baf366ca8b7"}], "block_device_mapping_v2": [{"uuid": "20fff726-13e1-4e4a-ab94-b75786fe7b1b", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:22.389596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] validate_networks() for [('66e518e7-6dfd-46cc-b9d0-1baf366ca8b7', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:49:23.147446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:49:23.150802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:49:23.150802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:49:23.150802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:49:23.150802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:49:23.158954 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:23.159659 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:23.160252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:23.161303 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:23.161933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:23.162541 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:23.189985 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Getting quotas for project 1330031e214045bf9d45c73da1445255. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:49:23.197965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Getting quotas for user 20bee9f38eb14541a393037eaa389a44 and project 1330031e214045bf9d45c73da1445255. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:49:23.364173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:49:23.366340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='20fff726-13e1-4e4a-ab94-b75786fe7b1b',volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:49:23.962643 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d1c86752-22b6-4b8d-b342-a4146030d8e0 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.853076 Sep 30 09:49:23.964069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 221/433] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:49:22 2022] POST /compute/v2.1/servers => generated 405 bytes in 1857 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:49:23.975124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7580953d-5055-4735-a295-c5dec20dc667 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:23.978018 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7580953d-5055-4735-a295-c5dec20dc667 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:24.090719 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7580953d-5055-4735-a295-c5dec20dc667 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:24.106275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7580953d-5055-4735-a295-c5dec20dc667 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1089 microversion: 2.1 time: 0.135400 Sep 30 09:49:24.118648 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 213/434] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:23 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1089 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:24.281708 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-58170280-60d5-4c0b-8874-174df60b49a4 req-ab02f913-410f-4b86-9b6a-840911e304dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35cb36cf-5501-4cf6-8a17-c988d7a0e337", "tag": "4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:24.295101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-58170280-60d5-4c0b-8874-174df60b49a4 req-ab02f913-410f-4b86-9b6a-840911e304dc service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:24.296846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-58170280-60d5-4c0b-8874-174df60b49a4 req-ab02f913-410f-4b86-9b6a-840911e304dc service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:24.297346 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-58170280-60d5-4c0b-8874-174df60b49a4 req-ab02f913-410f-4b86-9b6a-840911e304dc service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:24.319827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-58170280-60d5-4c0b-8874-174df60b49a4 req-ab02f913-410f-4b86-9b6a-840911e304dc service nova] Creating event network-changed:4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 for instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 on hv-cinder-maste Sep 30 09:49:24.327652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-58170280-60d5-4c0b-8874-174df60b49a4 req-ab02f913-410f-4b86-9b6a-840911e304dc service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054725 Sep 30 09:49:24.328590 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 222/435] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:25.150901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5342c9ed-7b70-4292-a7e3-0d3b44ee752e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:25.152097 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5342c9ed-7b70-4292-a7e3-0d3b44ee752e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:25.228616 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5342c9ed-7b70-4292-a7e3-0d3b44ee752e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:25.240613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5342c9ed-7b70-4292-a7e3-0d3b44ee752e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1089 microversion: 2.1 time: 0.112586 Sep 30 09:49:25.247884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 214/436] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:25 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1089 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:26.266950 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:26.268543 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:26.275419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:26.276145 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:26.277299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:26.278280 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:26.278920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:26.279649 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:26.359617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:26.373076 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-506bd003-4d7e-4d7f-b082-faa1abac65e4 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1187 microversion: 2.1 time: 0.112491 Sep 30 09:49:26.373651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 223/437] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:26 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1187 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:27.289000 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1664535141"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:27.289860 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:27.302890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:27.304047 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:27.305090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:27.310565 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:27.311083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:27.311515 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:27.402047 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:27.403797 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:27.410966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:27.411459 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:27.412114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:27.412809 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:27.413262 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:27.413857 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:27.518958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:27.527332 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-be5de9f5-9ecb-49a9-9ebf-1f8af23298e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1205 microversion: 2.1 time: 0.134510 Sep 30 09:49:27.532417 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 224/438] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:27 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1205 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:27.592809 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.network.security_group_api [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Adding security group a2f2441d-2469-4195-ba5b-d5e2f6ba3152 to port 4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 Sep 30 09:49:28.217925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1a2f1607-1e77-47ab-a26a-2cd502f9c25b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "POST /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/action" status: 202 len: 0 microversion: 2.1 time: 0.935820 Sep 30 09:49:28.218610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 215/439] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:49:27 2022] POST /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/action => generated 0 bytes in 938 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 30 09:49:28.226718 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:28.227336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:28.234290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:28.235996 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:28.237257 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:28.239539 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:28.239539 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:28.240178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:28.243405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-d9a501dc-6f7e-4446-8e86-bd775f80f0dd req-433d5a57-ab2d-4ad8-9ed2-935506c86add service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35cb36cf-5501-4cf6-8a17-c988d7a0e337", "tag": "4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:28.260949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d9a501dc-6f7e-4446-8e86-bd775f80f0dd req-433d5a57-ab2d-4ad8-9ed2-935506c86add service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:28.264114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d9a501dc-6f7e-4446-8e86-bd775f80f0dd req-433d5a57-ab2d-4ad8-9ed2-935506c86add service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:28.264761 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d9a501dc-6f7e-4446-8e86-bd775f80f0dd req-433d5a57-ab2d-4ad8-9ed2-935506c86add service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:28.294838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-d9a501dc-6f7e-4446-8e86-bd775f80f0dd req-433d5a57-ab2d-4ad8-9ed2-935506c86add service nova] Creating event network-changed:4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 for instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 on hv-cinder-maste Sep 30 09:49:28.300976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-d9a501dc-6f7e-4446-8e86-bd775f80f0dd req-433d5a57-ab2d-4ad8-9ed2-935506c86add service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061274 Sep 30 09:49:28.310703 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 216/440] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:28.574980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:28.574980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:28.574980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:28.574980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:28.579040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:28.580267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:28.581235 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:28.582415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:28.738180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:28.783253 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-509e9aec-477f-470d-8223-e9411bad87c1 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1760 microversion: 2.1 time: 0.560312 Sep 30 09:49:28.786380 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 225/441] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:28 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1760 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:28.968563 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:28.980115 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0c0d01b2-1fa9-42cb-b861-51d03559c8c5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1272 microversion: 2.1 time: 0.426163 Sep 30 09:49:28.992154 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 217/442] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:28 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1272 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:30.005096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:30.005723 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:30.014908 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:30.015399 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:30.020061 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:30.021348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:30.022290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:30.022762 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:30.256710 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:30.264802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a384f390-e3c8-49a5-b457-acf827468e8b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.264205 Sep 30 09:49:30.265801 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 226/443] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:29 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:30.347875 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-c43e4049-9755-4978-872a-b75b0bab2143 req-41315470-8f4f-4a66-93b5-2403d223c5bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dcd3d415-addc-4607-844d-2ab1c2c9d16b", "tag": "803fcd8d-66b3-48a0-bc79-84c2e79f336c"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:30.362298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=82,nova_api:UPDATE=2 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:30.364817 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-c43e4049-9755-4978-872a-b75b0bab2143 req-41315470-8f4f-4a66-93b5-2403d223c5bf service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:30.365484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-c43e4049-9755-4978-872a-b75b0bab2143 req-41315470-8f4f-4a66-93b5-2403d223c5bf service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:30.379438 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-c43e4049-9755-4978-872a-b75b0bab2143 req-41315470-8f4f-4a66-93b5-2403d223c5bf service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:30.422450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-c43e4049-9755-4978-872a-b75b0bab2143 req-41315470-8f4f-4a66-93b5-2403d223c5bf service nova] Creating event network-changed:803fcd8d-66b3-48a0-bc79-84c2e79f336c for instance dcd3d415-addc-4607-844d-2ab1c2c9d16b on hv-cinder-maste Sep 30 09:49:30.430201 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-c43e4049-9755-4978-872a-b75b0bab2143 req-41315470-8f4f-4a66-93b5-2403d223c5bf service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087396 Sep 30 09:49:30.431900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 218/444] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:31.289077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:31.290395 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:31.292826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:31.297675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:31.298394 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:31.299136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:31.300150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:31.300712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:31.301220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:31.539756 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:31.564652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3187ea24-0edd-409b-a9a3-34240fd7aa85 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.267948 Sep 30 09:49:31.564652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 227/445] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:31 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:32.578582 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:32.580043 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:32.586420 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:32.587041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:32.587709 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:32.588484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:32.589085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:32.589772 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:32.758258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:32.768199 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9527d4cd-7b8a-4294-933b-825cf8df5c65 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.194287 Sep 30 09:49:32.769145 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 219/446] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:32 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:33.792327 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:33.793280 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:33.800953 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:33.802899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=6 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:33.804444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:33.804948 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:33.805396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:33.806107 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:33.806626 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:33.807042 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:33.970437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:33.979391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c1ad2edc-995a-4604-90fd-5d247478ba0c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.190997 Sep 30 09:49:33.981685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 228/447] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:33 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:35.005219 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:35.005842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:35.012322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:35.012889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:35.013323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:35.014047 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:35.014496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:35.014947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:35.169603 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:35.177987 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:35.181612 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-05cb2607-7e9f-4336-8f2e-90e81a2ae1fd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.180604 Sep 30 09:49:35.182287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 220/448] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:34 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:36.212376 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:36.214051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:36.224356 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:36.225145 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:36.225883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:36.227312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:36.228320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:36.229670 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:36.400966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:36.410068 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e221313b-782a-4027-819e-9199462d63ea tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.207150 Sep 30 09:49:36.420009 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 229/449] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:36 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:37.433849 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:37.434698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:37.437024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=4 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:37.443393 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:37.444167 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:37.445052 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:37.446134 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:37.446843 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:37.447836 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:37.612859 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:37.622369 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9f542a30-b45e-422b-9d88-c17a185a657c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.192879 Sep 30 09:49:37.623013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 221/450] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:37 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:38.645284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:38.645897 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:38.653504 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:38.655233 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:38.655233 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:38.655571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:38.656134 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:38.656714 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:38.660718 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=91 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:49:38.793885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:38.802506 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2ba2535a-ac95-4a23-a59e-2bd140111b3d tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.161146 Sep 30 09:49:38.803127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 230/451] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:38 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:49:39.824277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:39.825158 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:39.831877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:39.832357 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:39.832786 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:39.833473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:39.834092 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:39.834509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:39.983562 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:39.992254 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-aac7e063-ae18-44a5-aa91-508780a6995a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.171821 Sep 30 09:49:39.994622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 222/452] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:39 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:40.015190 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-40c7b13d-9728-41bc-b79f-1d9d92ab19cb req-6edbc74e-26d9-4545-9bc7-ef5a6f96f2f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dcd3d415-addc-4607-844d-2ab1c2c9d16b", "name": "network-vif-plugged", "status": "completed", "tag": "803fcd8d-66b3-48a0-bc79-84c2e79f336c"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:40.025123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-40c7b13d-9728-41bc-b79f-1d9d92ab19cb req-6edbc74e-26d9-4545-9bc7-ef5a6f96f2f1 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:40.025604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-40c7b13d-9728-41bc-b79f-1d9d92ab19cb req-6edbc74e-26d9-4545-9bc7-ef5a6f96f2f1 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:40.026030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-40c7b13d-9728-41bc-b79f-1d9d92ab19cb req-6edbc74e-26d9-4545-9bc7-ef5a6f96f2f1 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:40.039085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-40c7b13d-9728-41bc-b79f-1d9d92ab19cb req-6edbc74e-26d9-4545-9bc7-ef5a6f96f2f1 service nova] Creating event network-vif-plugged:803fcd8d-66b3-48a0-bc79-84c2e79f336c for instance dcd3d415-addc-4607-844d-2ab1c2c9d16b on hv-cinder-maste Sep 30 09:49:40.044773 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-40c7b13d-9728-41bc-b79f-1d9d92ab19cb req-6edbc74e-26d9-4545-9bc7-ef5a6f96f2f1 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033202 Sep 30 09:49:40.045465 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 231/453] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:41.016261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:41.016861 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:41.023579 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:41.024056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:41.024488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:41.025166 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:41.025662 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:41.031151 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:41.157056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:41.164693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1afa8677-44a4-481a-aee9-5640f23d2cd7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.153108 Sep 30 09:49:41.168793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 223/454] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:41 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:42.188124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:42.188830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:42.195365 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:42.196069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:42.196500 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:42.197308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:42.197818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:42.198383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:42.333263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:42.341518 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b5b47514-e429-4ef3-b58f-359dacc8cf2b tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1260 microversion: 2.1 time: 0.158077 Sep 30 09:49:42.342079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 232/455] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:42 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1260 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:43.364202 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:43.364807 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:43.371766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:43.372241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:43.372673 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:43.373357 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:43.373802 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:43.374222 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:43.504803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:43.513190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-142f9d3a-7e11-40e2-a2ca-15b0bbed3576 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1444 microversion: 2.1 time: 0.152759 Sep 30 09:49:43.513759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 224/456] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:43 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1444 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:43.535588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:43.538428 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:43.542617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:43.543180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:43.543560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:43.544246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:43.544701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:43.545125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:43.679613 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:43.687500 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-78cdc5da-1dc4-478e-9095-53628b89ad87 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1444 microversion: 2.1 time: 0.155713 Sep 30 09:49:43.688257 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 233/457] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:43 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1444 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:45.609983 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f1609c14-b9aa-4f3c-956a-306e64f3ff52"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:45.609983 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:45.613759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:45.614264 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:45.614740 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:45.615431 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:45.615882 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:45.616301 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:45.824545 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-5712f97f-cbd8-4657-a285-e9cba5a06091 req-44df5d9c-0064-49b5-bf37-85f8f5728f8a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dcd3d415-addc-4607-844d-2ab1c2c9d16b", "tag": "803fcd8d-66b3-48a0-bc79-84c2e79f336c"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:45.836315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-5712f97f-cbd8-4657-a285-e9cba5a06091 req-44df5d9c-0064-49b5-bf37-85f8f5728f8a service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:45.836792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-5712f97f-cbd8-4657-a285-e9cba5a06091 req-44df5d9c-0064-49b5-bf37-85f8f5728f8a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:45.837220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-5712f97f-cbd8-4657-a285-e9cba5a06091 req-44df5d9c-0064-49b5-bf37-85f8f5728f8a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:45.848766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-5712f97f-cbd8-4657-a285-e9cba5a06091 req-44df5d9c-0064-49b5-bf37-85f8f5728f8a service nova] Creating event network-changed:803fcd8d-66b3-48a0-bc79-84c2e79f336c for instance dcd3d415-addc-4607-844d-2ab1c2c9d16b on hv-cinder-maste Sep 30 09:49:45.853982 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-5712f97f-cbd8-4657-a285-e9cba5a06091 req-44df5d9c-0064-49b5-bf37-85f8f5728f8a service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033690 Sep 30 09:49:45.854644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 234/458] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:49:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:46.435722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2f80c4a9-1930-4fbe-92d6-d703c3bd25d9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.835242 Sep 30 09:49:46.436764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 225/459] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:49:45 2022] POST /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments => generated 194 bytes in 837 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:49:51.859864 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:51.862120 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:51.869354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:51.869906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:51.870411 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:51.871246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:51.871705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:51.872128 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:52.064289 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ab20aa68-7ad3-4df0-b912-4d838ae6f5f5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "POST /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/action" status: 202 len: 0 microversion: 2.1 time: 0.209464 Sep 30 09:49:52.065325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 235/460] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:49:51 2022] POST /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/action => generated 0 bytes in 212 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:49:52.074894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:52.075691 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:52.085027 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:52.085648 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:52.086050 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:52.086783 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:52.087236 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:52.087658 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:52.506650 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:52.515287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-675e027e-aa7d-419d-a09c-d57b864f759f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1752 microversion: 2.1 time: 0.445124 Sep 30 09:49:52.515858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 226/461] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:52 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1752 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:53.541774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:53.542465 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:53.549059 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:53.549540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:53.549966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:53.550678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:53.551155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:53.551571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:53.708769 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:53.717289 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-44a3bec8-8a72-4d20-a912-fa17e538975f tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1752 microversion: 2.1 time: 0.179709 Sep 30 09:49:53.718140 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 236/462] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:53 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1752 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:54.741481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:54.742232 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:54.749133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:54.750390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:54.750390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:54.750895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:54.751343 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:54.751758 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:54.900429 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:54.909238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6b83e01a-ef04-42ed-8f61-d6b921b3a5f9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.171784 Sep 30 09:49:54.909798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 227/463] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:54 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:55.067228 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:49:55.069432 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:55.075933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:55.076425 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:55.076863 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:55.077589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:55.078045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:55.079079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:55.104186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lazy-loading 'flavor' on Instance uuid 44e7298b-38f3-436b-98ed-e9157faaca5e {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:49:55.441698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-6f4fbc33-da0d-4da2-8ff3-2fe71cacd004 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/action" status: 200 len: 32 microversion: 2.1 time: 0.379162 Sep 30 09:49:55.442359 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 237/464] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:49:55 2022] POST /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/action => generated 32 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 09:49:55.451848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:55.452658 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:55.459127 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:55.463485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:55.463485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:55.463485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:55.463485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:55.463947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:55.592301 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:55.601144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9dfc2d73-6d4e-44c7-8394-927595a7aea0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.153126 Sep 30 09:49:55.601702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 228/465] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:55 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:55.935780 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:55.936475 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:55.942904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:55.943397 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:55.943828 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:55.944521 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:55.944965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:55.945384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:56.187105 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:56.195538 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dc4bda5e-8f4d-4bd8-b9b5-7f316379acd9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.263958 Sep 30 09:49:56.196260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 238/466] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:55 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:56.624901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:56.626609 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:56.633230 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:56.633710 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:56.636926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:56.636926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:56.636926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:56.636926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:56.767063 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:56.775649 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e85be56a-5d72-4c8e-907a-96d045a5c980 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.154584 Sep 30 09:49:56.776217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 229/467] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:56 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:57.219182 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:57.219806 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:57.226112 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:57.226607 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:57.227060 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:57.227744 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:57.228196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:57.228615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:57.375953 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:57.384672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d5979086-bd4b-498a-b002-638a4ebf9aa8 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.169309 Sep 30 09:49:57.385345 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 239/468] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:57 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:57.798660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:57.799247 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:57.806483 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:57.807097 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:57.807532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:57.808241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:57.808692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:57.809106 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:57.939575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:57.949513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9d84bc44-0d5c-49cd-98b9-6b3a35958acc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.154697 Sep 30 09:49:57.950089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 230/469] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:57 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:58.407540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:58.408150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:58.414459 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:58.414955 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:58.415391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:58.416081 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:58.416535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:58.416965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:58.560496 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:58.569197 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-39607eeb-796e-4a1a-bf43-14ac8155b748 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.165505 Sep 30 09:49:58.569753 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 240/470] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:58 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:58.973490 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:58.974093 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:58.980499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:58.980989 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:58.981425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:58.982113 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:58.982563 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:58.983010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:59.125582 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:59.134040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-02716f48-f799-4407-9cbf-fc98f9ea86a7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.164991 Sep 30 09:49:59.135017 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 231/471] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:58 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:49:59.592136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:49:59.592739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:49:59.599228 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:59.599712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:59.600143 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:59.600828 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:49:59.601281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:49:59.601699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:49:59.739409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:49:59.747889 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0d2bbe1d-b406-41de-8560-a23b361f0ad9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.159642 Sep 30 09:49:59.748455 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 241/472] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:49:59 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:00.157683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:00.158310 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:00.164944 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.165420 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.165844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.166522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.166993 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.167408 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.301906 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:00.310185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-71846c2e-4c74-4ff5-a8f7-71affa80d4c8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.156806 Sep 30 09:50:00.310765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 232/473] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:00 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:00.744940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:00.745452 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:00.751752 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.752265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.752727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.753415 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.753898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.754324 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.772012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:00.772699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:00.777937 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7f43a234-d74e-4d73-a2ce-5c2b4d61dc44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed/ips" status: 200 len: 150 microversion: 2.1 time: 0.037147 Sep 30 09:50:00.778523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 242/474] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 09:50:00 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed/ips => generated 150 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:00.778679 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.779131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.779966 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.780671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.781172 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.782237 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.790334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c3971c02-ab43-47a0-bad0-be731ce4d4be"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:00.792846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:00.798805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.799297 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.799735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.800430 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.800886 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.801315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.880393 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Invalid volume: volume c3971c02-ab43-47a0-bad0-be731ce4d4be is already attached to instances: 0bc1e701-2247-4837-81c0-289806279c31 Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: POST /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed/os-volume_attachments HTTP/1.1 Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 74 Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:50:00.882118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {"volumeAttachment": {"volumeId": "c3971c02-ab43-47a0-bad0-be731ce4d4be"}} Sep 30 09:50:00.883640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Invalid volume: volume c3971c02-ab43-47a0-bad0-be731ce4d4be is already attached to instances: 0bc1e701-2247-4837-81c0-289806279c31 Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'server_id': '59975d37-8db0-4019-9303-88b12ef029ed', 'body': {'volumeAttachment': {'volumeId': 'c3971c02-ab43-47a0-bad0-be731ce4d4be'}}} Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume c3971c02-ab43-47a0-bad0-be731ce4d4be is already attached to instances: 0bc1e701-2247-4837-81c0-289806279c31 Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:00.884419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume c3971c02-ab43-47a0-bad0-be731ce4d4be is already attached to instances: 0bc1e701-2247-4837-81c0-289806279c31 Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:00.885892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:50:00.887466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:50:00.887466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:50:00.887466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:50:00.887466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume c3971c02-ab43-47a0-bad0-be731ce4d4be is already attached to instances: 0bc1e701-2247-4837-81c0-289806279c31 Sep 30 09:50:00.887466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:00.887466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 400 to user: Invalid volume: volume c3971c02-ab43-47a0-bad0-be731ce4d4be is already attached to instances: 0bc1e701-2247-4837-81c0-289806279c31 {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:50:00.887466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-93e78aff-1bd2-4982-9109-a34791d3b3e8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.097111 Sep 30 09:50:00.887466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 243/475] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:50:00 2022] POST /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed/os-volume_attachments => generated 174 bytes in 98 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 30 09:50:00.892523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:00.893012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:00.902070 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.902573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.903094 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.903801 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:00.904262 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:00.904689 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:00.927940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:00.930394 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:50:00.936133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5c570abf-5e7f-4cb8-a68c-325dc6cf61eb tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.168155 Sep 30 09:50:00.936694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 233/476] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:00 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:00.955069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:01.202939 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1bbefcd8-f02f-482a-b099-037ef88ee3df tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 204 len: 0 microversion: 2.1 time: 0.314175 Sep 30 09:50:01.203618 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 244/477] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:50:00 2022] DELETE /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 0 bytes in 316 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:50:01.212453 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:01.213054 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:01.219618 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:01.220103 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:01.220535 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:01.221227 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:01.221693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:01.222100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:01.334305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:01.335035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:01.340859 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:01.341325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:01.341756 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:01.342443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:01.342929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:01.343347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:01.352835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:01.361019 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c3f1639c-34dd-4c54-b5d4-0be6a2e8ae28 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1678 microversion: 2.1 time: 0.151967 Sep 30 09:50:01.361588 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 234/478] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:01 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1678 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:01.469477 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:01.477562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-64bb7e6a-0ce6-4d14-bbfa-22493b81073c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.147980 Sep 30 09:50:01.478113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 245/479] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:01 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:01.959653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:01.960257 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:01.966036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:01.966508 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:01.966963 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:01.967638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:01.968089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:01.968502 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:02.105801 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:02.114970 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3b8fa16d-1263-443e-9170-b814a914d867 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.158808 Sep 30 09:50:02.115529 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 235/480] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:01 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:02.386092 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:02.386786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:02.389105 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:02.394219 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:02.394725 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:02.395155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:02.395843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:02.396285 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:02.396700 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:02.501496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:02.502710 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:02.509159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:02.509641 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:02.510073 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:02.510806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:02.511262 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:02.511684 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:02.516664 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:02.529180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:02.537991 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-773cc8de-cf59-4b2b-822b-a283ac9fd340 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1678 microversion: 2.1 time: 0.155653 Sep 30 09:50:02.538560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 246/481] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:02 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1678 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:02.657486 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:02.657486 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0312abe7-eb90-4535-87a2-93dc223063e9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.157033 Sep 30 09:50:02.657486 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 236/482] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:02 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:03.138312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:03.138952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:03.144794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:03.145267 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:03.145694 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:03.146371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:03.146846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:03.147266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:03.302747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:03.310918 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-df5f7990-5a0e-44f6-9bd0-d724599882b0 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.176382 Sep 30 09:50:03.311479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 247/483] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:03 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:03.563954 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:03.564571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:03.567217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=38 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:03.573020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:03.574190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:03.575079 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:03.576032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:03.578329 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:03.579063 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:03.677524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:03.686784 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:03.692830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:03.693329 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:03.693751 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:03.694440 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:03.694938 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:03.705568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:03.799085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:03.807469 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b694c551-a25f-4b2c-816d-9cd33077ffe7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1678 microversion: 2.1 time: 0.249765 Sep 30 09:50:03.808048 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 237/484] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:03 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1678 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:03.956818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:03.965458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-415a4cfa-c4b2-44ea-a18c-cf331c3218cb tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.291792 Sep 30 09:50:03.981090 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 248/485] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:03 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:04.334241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:04.334371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:04.342534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:04.342534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:04.342534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:04.342995 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:04.343929 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:04.343929 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:04.455212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-0c826c7e-64d9-4354-bff6-292be4ab6821 req-a67de4f2-d8db-4cbd-936a-c6a5cd4bd330 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59975d37-8db0-4019-9303-88b12ef029ed", "name": "network-vif-deleted", "tag": "b4db42a3-b3ea-4c10-8fa2-c3c219dd729e"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:04.464901 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0c826c7e-64d9-4354-bff6-292be4ab6821 req-a67de4f2-d8db-4cbd-936a-c6a5cd4bd330 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:04.469099 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0c826c7e-64d9-4354-bff6-292be4ab6821 req-a67de4f2-d8db-4cbd-936a-c6a5cd4bd330 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:04.469099 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0c826c7e-64d9-4354-bff6-292be4ab6821 req-a67de4f2-d8db-4cbd-936a-c6a5cd4bd330 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:04.477788 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-0c826c7e-64d9-4354-bff6-292be4ab6821 req-a67de4f2-d8db-4cbd-936a-c6a5cd4bd330 service nova] Creating event network-vif-deleted:b4db42a3-b3ea-4c10-8fa2-c3c219dd729e for instance 59975d37-8db0-4019-9303-88b12ef029ed on hv-cinder-maste Sep 30 09:50:04.484404 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-0c826c7e-64d9-4354-bff6-292be4ab6821 req-a67de4f2-d8db-4cbd-936a-c6a5cd4bd330 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032993 Sep 30 09:50:04.486072 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 249/486] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:50:04.505958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:04.519441 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4ca9d3df-6cb8-4d60-9f25-9d41f78ad241 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.189964 Sep 30 09:50:04.520287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 238/487] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:04 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:04.832427 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:04.833191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:04.839536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:04.840019 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:04.847361 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:04.848461 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:04.849516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:04.850162 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:04.886565 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:04.887305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:04.887885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:04.888497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lazy-loading 'fault' on Instance uuid 59975d37-8db0-4019-9303-88b12ef029ed {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:50:04.947065 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:04.955187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c858663e-e5f7-4a7c-a373-fb0aa1333178 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 200 len: 1328 microversion: 2.1 time: 0.128216 Sep 30 09:50:04.955744 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 250/488] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:04 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 1328 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:04.990267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:04.991190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:04.999044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:05.000648 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:05.001037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:05.002258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:05.002854 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:05.003380 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:05.155654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:05.164864 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c3d2a718-5d48-403b-9fec-1233d1ac5769 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.179507 Sep 30 09:50:05.165511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 239/489] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:04 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:05.546563 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:05.546563 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:05.553085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:05.553703 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:05.554232 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:05.555055 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:05.555737 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:05.556371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:05.714744 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:05.723243 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f88c80ce-d454-44ab-8cba-b8921715ec5c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.183653 Sep 30 09:50:05.723884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 251/490] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:05 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:05.984611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:05.985200 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:05.992008 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:05.992622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:05.993057 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:05.993741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:05.994197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:05.994912 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:06.009714 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed HTTP/1.1 Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:50:06.011415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:50:06.013126 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '59975d37-8db0-4019-9303-88b12ef029ed'} Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:50:06.013940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:50:06.015377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:06.016538 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:50:06.016538 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f2580c8f-8f95-410f-9775-834af6d4c4b7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 404 len: 111 microversion: 2.1 time: 0.032208 Sep 30 09:50:06.016538 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 240/491] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:05 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:50:06.021866 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-74e3435e-3837-4ba3-89d0-433571457b74 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '.version_select at 0x7f72c199de50>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:06.104864 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-74e3435e-3837-4ba3-89d0-433571457b74 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2192127" status: 202 len: 0 microversion: 2.1 time: 0.086808 Sep 30 09:50:06.105381 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 252/492] 10.222.0.39 () {58 vars in 1212 bytes} [Fri Sep 30 09:50:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2192127 => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:50:06.200848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:06.201762 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:06.208591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:06.209184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:06.209685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:06.210545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:06.211163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:06.211692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:06.394528 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:06.403223 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ef9bc04e-e87a-401d-98f9-9fe28ef901f9 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.214723 Sep 30 09:50:06.403831 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 241/493] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:06 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:06.747238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:06.748041 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:06.754959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:06.755450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:06.755887 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:06.756593 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:06.757050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:06.757477 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:06.900329 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:06.909990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-eddc81f7-024b-4a19-94e0-e75fe2b9eb56 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.166816 Sep 30 09:50:06.916294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 253/494] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:06 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:07.160381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:07.160850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:07.167097 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:07.167574 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:07.167994 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:07.168670 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:07.169111 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:07.169525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:07.291901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lazy-loading 'services' on Instance uuid 0bc1e701-2247-4837-81c0-289806279c31 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:50:07.427543 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:07.428084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:07.434402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:07.434913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:07.435348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:07.436030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:07.436480 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:07.436902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:07.549212 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-026918f5-0299-4e81-97c1-d6a306631eca tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments/c3971c02-ab43-47a0-bad0-be731ce4d4be" status: 202 len: 0 microversion: 2.1 time: 0.391977 Sep 30 09:50:07.549932 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 242/495] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:50:07 2022] DELETE /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments/c3971c02-ab43-47a0-bad0-be731ce4d4be => generated 0 bytes in 394 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:50:07.558525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:07.560851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:07.568415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:07.569034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:07.569470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:07.570181 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:07.570668 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:07.571097 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:07.606850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b21c30bc-0cc0-48a3-9684-fa690d6bec23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052149 Sep 30 09:50:07.607430 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 243/496] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:50:07 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:07.632621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:07.640738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9c019c2c-952f-4c65-b005-93aadea5e90c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.217706 Sep 30 09:50:07.641288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 254/497] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:07 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:07.933203 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:07.933803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:07.940397 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:07.940872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:07.941300 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:07.941978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:07.942421 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:07.942869 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:08.083932 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:08.092009 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-cf7452ca-094b-4989-bd91-2778d9403ade tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.162690 Sep 30 09:50:08.092570 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 244/498] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:07 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:08.620304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:08.622553 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:08.629544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:08.630019 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:08.630444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:08.631169 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:08.631617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:08.632030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:08.661668 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1f6de11a-fd13-430d-a94c-948e111d4033 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045596 Sep 30 09:50:08.662228 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 255/499] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:50:08 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:08.662997 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:08.663605 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:08.669931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:08.670408 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:08.670871 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:08.671561 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:08.672012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:08.672430 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:08.797824 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:08.805949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2a408dba-610b-4fd1-be98-b1433de15573 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.146813 Sep 30 09:50:08.806517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 245/500] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:08 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:09.115965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:09.116575 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:09.122904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:09.123383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:09.123815 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:09.124499 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:09.124955 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:09.125383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:09.265009 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:09.273139 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ff1619d9-5cfb-40d4-b0a0-235bb0b27b2c tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.161059 Sep 30 09:50:09.273700 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 256/501] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:09 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:09.673375 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:09.675801 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:09.682622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:09.683110 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:09.684105 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:09.684830 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:09.685322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:09.685778 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:09.715316 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e449ce6f-7529-40b3-8adc-f275bbc39cf5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045816 Sep 30 09:50:09.715871 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 246/502] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:50:09 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:09.829494 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:09.830678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:09.836987 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:09.837471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:09.837908 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:09.838616 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:09.839077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:09.839502 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:09.968283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:09.977143 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ebb23146-dfc2-4e20-a128-baa41a5587ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.151622 Sep 30 09:50:09.977709 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 257/503] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:09 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:10.296418 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:10.297024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:10.303505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:10.303981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:10.304410 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:10.305124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:10.305579 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:10.305998 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:10.445084 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:10.453426 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-44b2b686-084a-434e-8893-a692dde2294b tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1757 microversion: 2.1 time: 0.160728 Sep 30 09:50:10.453960 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 247/504] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:10 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1757 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:10.727591 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:10.729799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:10.736605 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:10.737097 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:10.737548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:10.738239 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:10.738727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:10.739158 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:10.769520 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-53a9f6be-d6c6-4806-bd31-fd1a841769af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045857 Sep 30 09:50:10.770067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 258/505] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:50:10 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:11.000022 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:11.000641 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:11.007637 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:11.011034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:11.011034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:11.011034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:11.011034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:11.011457 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:11.145423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:11.153710 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-038fe3fb-7b26-42a9-97c9-0ff47f638f73 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.157651 Sep 30 09:50:11.154269 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 248/506] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:10 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:11.477548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:11.478154 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:11.484685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:11.485166 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:11.485599 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:11.486949 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:11.487403 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:11.487824 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:11.624620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:11.632670 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1ba8334e-8f0c-4028-811b-63e17ed9e312 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1760 microversion: 2.1 time: 0.158996 Sep 30 09:50:11.633232 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 259/507] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:11 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1760 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:11.783524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:11.785745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:11.791739 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:11.792221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:11.792739 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:11.793357 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:11.793804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:11.794221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:11.823121 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c36639ed-5e37-4860-9ce6-ef0656f11c6e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044689 Sep 30 09:50:11.828986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 249/508] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:50:11 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 09:50:11.888273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:11.888956 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:11.896680 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:11.897158 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:11.897595 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:11.898280 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:11.898793 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:11.899211 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:11.926049 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:50:11.957970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:12.177075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:12.177683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:12.184103 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:12.184585 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:12.185017 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:12.185724 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:12.186169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:12.186572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:12.189222 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-23c8fe5e-20b2-4021-8233-bb48d3afb537 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 204 len: 0 microversion: 2.1 time: 0.306030 Sep 30 09:50:12.190041 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 260/509] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:50:11 2022] DELETE /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:50:12.198540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:12.205625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:12.205625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:12.206487 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:12.206661 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:12.207352 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:12.207869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:12.208774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:12.385454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:12.395694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8854f784-05ad-4911-8f69-b8b3aca117da tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.221705 Sep 30 09:50:12.404192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:12.407405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 250/510] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:12 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:12.413890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b5dfc48a-1fd0-4f05-9019-7d51d376c7a1 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1683 microversion: 2.1 time: 0.219412 Sep 30 09:50:12.414538 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 261/511] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:12 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1683 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:13.417523 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:13.418120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:13.424417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:13.424891 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:13.425319 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:13.426030 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:13.426473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:13.426913 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:13.437102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:13.439253 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:13.448698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:13.448698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:13.448698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:13.448698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:13.449048 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:13.449048 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:13.632876 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:13.637113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:13.641144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-48e97597-3a3d-4f5f-8707-409563290568 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.227486 Sep 30 09:50:13.642905 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 251/512] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:13 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:13.645445 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2fe23223-e630-49a3-8c6b-160bfc8e31c2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1683 microversion: 2.1 time: 0.212188 Sep 30 09:50:13.645998 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 262/513] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:13 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1683 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:14.665208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:14.665879 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:14.668948 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:14.669550 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:14.672318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:14.672803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:14.673231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:14.674572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:14.675053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:14.675473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:14.676591 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:14.677064 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:14.677495 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:14.678180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:14.678655 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:14.679076 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:15.067964 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:15.070228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:15.076822 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-93142286-bdc2-4fac-8d83-2fcacb66da7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.415461 Sep 30 09:50:15.077395 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 252/514] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:14 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:15.078978 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:15.087244 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-274fa51d-8c16-45a9-9d06-90ffdc53e574 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1683 microversion: 2.1 time: 0.422047 Sep 30 09:50:15.087799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 263/515] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:14 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1683 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:16.102309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:16.102858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:16.109558 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:16.110041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:16.110385 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:16.110842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:16.111076 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:16.111800 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:16.112252 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:16.112670 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:16.118250 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:16.118765 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:16.119204 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:16.120174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:16.120686 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:16.121111 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:16.304049 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:16.310619 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:16.311785 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f79937d5-2d72-4348-bf26-4127058e7ce7 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1683 microversion: 2.1 time: 0.205182 Sep 30 09:50:16.312575 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 264/516] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:16 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1683 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:16.318987 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c17fe991-fae7-408d-bc4e-d08bccbecc10 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.220980 Sep 30 09:50:16.319549 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 253/517] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:16 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:17.335312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:17.335935 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:17.342059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:17.342536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:17.342839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:17.343023 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:17.343458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:17.344251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:17.344615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:17.345058 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:17.349867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:17.350461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:17.350816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:17.352388 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:17.352388 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:17.352388 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:17.543000 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:17.546516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:17.551366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9e5f126d-bebe-4184-afb0-52a41829a2e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1683 microversion: 2.1 time: 0.219817 Sep 30 09:50:17.552132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 265/518] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:17 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1683 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:17.555660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-53f062b0-801d-44d0-a409-d354dc1de4c4 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.217381 Sep 30 09:50:17.565180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 254/519] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:17 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:17.573865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-7b9b77fc-b421-48e5-aec8-3d152b216ebd req-c686c1cd-b6d8-4326-ab0e-99c9cccfefc3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0bc1e701-2247-4837-81c0-289806279c31", "name": "network-vif-unplugged", "status": "completed", "tag": "cc523046-219f-42b7-a7b9-700936140c81"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:17.581966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7b9b77fc-b421-48e5-aec8-3d152b216ebd req-c686c1cd-b6d8-4326-ab0e-99c9cccfefc3 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:17.582438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7b9b77fc-b421-48e5-aec8-3d152b216ebd req-c686c1cd-b6d8-4326-ab0e-99c9cccfefc3 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:17.582904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7b9b77fc-b421-48e5-aec8-3d152b216ebd req-c686c1cd-b6d8-4326-ab0e-99c9cccfefc3 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:17.595942 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-7b9b77fc-b421-48e5-aec8-3d152b216ebd req-c686c1cd-b6d8-4326-ab0e-99c9cccfefc3 service nova] Creating event network-vif-unplugged:cc523046-219f-42b7-a7b9-700936140c81 for instance 0bc1e701-2247-4837-81c0-289806279c31 on hv-cinder-maste Sep 30 09:50:17.600657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-7b9b77fc-b421-48e5-aec8-3d152b216ebd req-c686c1cd-b6d8-4326-ab0e-99c9cccfefc3 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.033351 Sep 30 09:50:17.601231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 266/520] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:17 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:18.479228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-928b5202-ca90-41a4-ba66-285e4c8af2ac req-51014ae9-da0d-429e-82f0-809a2298b948 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0bc1e701-2247-4837-81c0-289806279c31", "name": "network-vif-deleted", "tag": "cc523046-219f-42b7-a7b9-700936140c81"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:18.489980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-928b5202-ca90-41a4-ba66-285e4c8af2ac req-51014ae9-da0d-429e-82f0-809a2298b948 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:18.490454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-928b5202-ca90-41a4-ba66-285e4c8af2ac req-51014ae9-da0d-429e-82f0-809a2298b948 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:18.494040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-928b5202-ca90-41a4-ba66-285e4c8af2ac req-51014ae9-da0d-429e-82f0-809a2298b948 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:18.502422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-928b5202-ca90-41a4-ba66-285e4c8af2ac req-51014ae9-da0d-429e-82f0-809a2298b948 service nova] Creating event network-vif-deleted:cc523046-219f-42b7-a7b9-700936140c81 for instance 0bc1e701-2247-4837-81c0-289806279c31 on hv-cinder-maste Sep 30 09:50:18.507882 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-928b5202-ca90-41a4-ba66-285e4c8af2ac req-51014ae9-da0d-429e-82f0-809a2298b948 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032624 Sep 30 09:50:18.508747 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 255/521] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:18.581482 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:18.582883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:18.583492 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:18.585205 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:18.590695 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:18.591432 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:18.592103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:18.592975 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:18.593595 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:18.594178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:18.613660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:18.618643 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:18.619150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:18.619890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:18.623374 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:18.624175 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:18.692189 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:18.700470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e1896026-6357-4fa9-aa83-7e194b5cd163 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1615 microversion: 2.1 time: 0.128911 Sep 30 09:50:18.701055 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 267/522] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:18 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1615 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:18.805762 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:18.814960 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2d25101f-77cc-45f4-89c6-cb67db5d072f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1736 microversion: 2.1 time: 0.237621 Sep 30 09:50:18.815648 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 256/523] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:18 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1736 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:19.726132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:19.727043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:19.734386 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:19.734910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:19.735347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:19.736225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:19.736690 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:19.737117 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:19.769598 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:19.770082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:19.770506 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:19.770971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lazy-loading 'fault' on Instance uuid 0bc1e701-2247-4837-81c0-289806279c31 {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:50:19.812038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:19.820440 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-99e63a74-f58d-4863-a003-db9a6b96033e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 200 len: 1332 microversion: 2.1 time: 0.098864 Sep 30 09:50:19.821011 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 268/524] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:19 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 1332 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:19.838758 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:19.839559 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:19.846678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:19.847309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:19.847749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:19.848488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:19.848941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:19.849365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:19.980947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:19.989125 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6991df61-d9bb-4e76-b0bd-3a279911e2ae tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1716 microversion: 2.1 time: 0.154866 Sep 30 09:50:19.991007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 257/525] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:19 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1716 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:20.100524 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:20.101101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:20.107031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:20.107520 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:20.107960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:20.108655 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:20.109114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:20.109542 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:20.236638 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] HTTP exception thrown: Cannot 'detach_volume' instance 44e7298b-38f3-436b-98ed-e9157faaca5e while it is in vm_state rescued Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Request method failure captured: Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments/f1609c14-b9aa-4f3c-956a-306e64f3ff52 HTTP/1.1 Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrTlXnTtrSqp1V_x1W1vnIRpWmkBjrFT8cNypZpVQr27Bt1ba58D9-pK8ZLi7HjxFCHn6nlNUOFaxuG8SOFGbJyOl9-u5Y9sbOG4I2nKOR06Qh9xhjqVXK1QVMfkONcO4rT4iNkpR9PqGmjOgNcaMeH-UkdqodqHrtoefVyVzzrL0A Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:50:20.238281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 7ad21da05fc246d489b0afa641f20216 Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:50:20.239929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Cannot 'detach_volume' instance 44e7298b-38f3-436b-98ed-e9157faaca5e while it is in vm_state rescued Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'server_id': '44e7298b-38f3-436b-98ed-e9157faaca5e', 'id': 'f1609c14-b9aa-4f3c-956a-306e64f3ff52'} Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 44e7298b-38f3-436b-98ed-e9157faaca5e in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:50:20.240740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 44e7298b-38f3-436b-98ed-e9157faaca5e while it is in vm_state rescued Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 44e7298b-38f3-436b-98ed-e9157faaca5e while it is in vm_state rescued Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.241984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Returning 409 to user: Cannot 'detach_volume' instance 44e7298b-38f3-436b-98ed-e9157faaca5e while it is in vm_state rescued {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:50:20.243031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8c9a6e77-7245-492f-ac33-b6ab58bd2352 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "DELETE /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments/f1609c14-b9aa-4f3c-956a-306e64f3ff52" status: 409 len: 152 microversion: 2.1 time: 0.143265 Sep 30 09:50:20.243031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 269/526] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:50:20 2022] DELETE /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments/f1609c14-b9aa-4f3c-956a-306e64f3ff52 => generated 152 bytes in 144 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 30 09:50:20.249135 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:20.249763 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:20.256359 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:20.256839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:20.257266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:20.257941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:20.258384 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:20.258830 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:20.507034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-708bafc7-5b84-452e-8cce-846236c55791 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "POST /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/action" status: 202 len: 0 microversion: 2.1 time: 0.261708 Sep 30 09:50:20.507839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 258/527] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:50:20 2022] POST /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/action => generated 0 bytes in 263 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:50:20.516792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:20.517406 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:20.523437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:20.523927 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:20.524364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:20.525060 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:20.525514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:20.525936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:20.653336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:20.662550 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-28ce9c98-997f-446f-a72f-401e928b95f2 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1724 microversion: 2.1 time: 0.150475 Sep 30 09:50:20.663150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 270/528] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:20 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1724 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:20.849130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:20.849861 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:20.855905 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:20.856381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:20.856809 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:20.857496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:20.857948 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:20.858372 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:20.871891 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 HTTP/1.1 Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:50:20.873499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:50:20.874986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '0bc1e701-2247-4837-81c0-289806279c31'} Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:50:20.875838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:50:20.877258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:50:20.878539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:50:20.878539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-38f1bd3b-d411-4e9a-8417-32f1897d0102 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 404 len: 111 microversion: 2.1 time: 0.031029 Sep 30 09:50:20.878539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 259/529] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:20 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:50:20.883622 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f61aac6f-3abc-43ec-8135-182dd786443f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '.version_select at 0x7f72c0b0a4c0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:20.976270 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f61aac6f-3abc-43ec-8135-182dd786443f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2119459492" status: 202 len: 0 microversion: 2.1 time: 0.096366 Sep 30 09:50:20.976862 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 271/530] 10.222.0.39 () {58 vars in 1221 bytes} [Fri Sep 30 09:50:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2119459492 => generated 0 bytes in 98 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:50:21.686344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:21.687063 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:21.693630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:21.694184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:21.694741 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:21.695435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:21.695884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:21.696298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:21.901278 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:21.909992 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8147a4b9-43a4-4032-aec6-1836621a499a tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1724 microversion: 2.1 time: 0.227441 Sep 30 09:50:21.910808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 260/531] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:21 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1724 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:21.940317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5b90c357-25cf-4139-921e-5459e945e929 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: .version_select at 0x7f72c0ace1f0>, body: {"keypair": {"name": "tempest-keypair-91522501", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIBaQSPSOtiJP9UHJ0swUuUr0OWfVsK0eWogVvXLHfqVDO/5S2TZku6bPZbC5PFf54MWyL9IFeYO90WE0BCPYNotYtQSdPLh7yntBRplHhgKklwa4tzjiTB9iMZlzlWnaA=="}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:21.956619 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-5b90c357-25cf-4139-921e-5459e945e929 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for project 86e6b0501cb94028be07bddd2182638b. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:50:21.960528 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-5b90c357-25cf-4139-921e-5459e945e929 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for user 8c4ced79c3ff4842b30b472acb555cd5 and project 86e6b0501cb94028be07bddd2182638b. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:50:22.038696 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5b90c357-25cf-4139-921e-5459e945e929 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.103446 Sep 30 09:50:22.039287 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 272/532] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:50:21 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:50:22.935622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:22.936227 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:22.943196 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:22.943702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:22.944466 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:22.945425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:22.945866 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:22.946629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:23.090161 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:23.098536 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-74db8bb4-66d2-4770-ad64-7a36d6c65dab tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1724 microversion: 2.1 time: 0.166739 Sep 30 09:50:23.099129 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 261/533] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:22 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1724 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:24.123742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:24.124428 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:24.134265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:24.134265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:24.134265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:24.134265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:24.134581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:24.134581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:24.236791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1658915250", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d038a068-f643-497f-8e6f-7d6e8fb5892b"}], "security_groups": [{"name": "tempest-securitygroup--351958239"}], "key_name": "tempest-keypair-91522501", "user_data": "}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:24.276180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:24.284880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-035dabb7-0f18-4710-b135-fa4f684ce372 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1730 microversion: 2.1 time: 0.166054 Sep 30 09:50:24.298578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 273/534] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:24 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1730 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:24.308967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:24.309968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:24.317654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:24.318275 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:24.318636 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:24.319326 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:24.319781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:24.320203 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:24.351778 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] validate_networks() for [('d038a068-f643-497f-8e6f-7d6e8fb5892b', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:50:24.457611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3d532fe5-fc72-46c8-909e-f76381906c5e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.152682 Sep 30 09:50:24.458344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 274/535] 10.222.0.39 () {58 vars in 1275 bytes} [Fri Sep 30 09:50:24 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-interface => generated 284 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:50:24.789449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:50:24.789968 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:50:24.790654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:50:24.791107 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:50:24.791522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:50:24.805007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:24.805007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:24.805007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:24.805007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:24.805607 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:24.805607 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:24.822557 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for project 86e6b0501cb94028be07bddd2182638b. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:50:24.826826 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for user 8c4ced79c3ff4842b30b472acb555cd5 and project 86e6b0501cb94028be07bddd2182638b. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:50:24.838654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:50:24.841004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:50:24.973211 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c46bd04b-2f44-4e2f-8c56-df3edc1f0783 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.740738 Sep 30 09:50:24.974387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 262/536] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:50:24 2022] POST /compute/v2.1/servers => generated 401 bytes in 742 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:50:24.985831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fb8f5e13-3dc0-4d71-bc44-a8c2f8af7c58 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:24.988288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fb8f5e13-3dc0-4d71-bc44-a8c2f8af7c58 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:25.066751 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fb8f5e13-3dc0-4d71-bc44-a8c2f8af7c58 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:25.073481 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fb8f5e13-3dc0-4d71-bc44-a8c2f8af7c58 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1240 microversion: 2.1 time: 0.091680 Sep 30 09:50:25.074134 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 275/537] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:24 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1240 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:25.482738 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-477ed117-f33a-4187-9abd-adc128bbcb41 req-638d44ce-851d-4c7c-926e-312b4eca0508 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "44e7298b-38f3-436b-98ed-e9157faaca5e", "tag": "411b31d5-4a9b-44b2-8ce8-22d9cb075ef9"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:25.493723 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-477ed117-f33a-4187-9abd-adc128bbcb41 req-638d44ce-851d-4c7c-926e-312b4eca0508 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:25.494289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-477ed117-f33a-4187-9abd-adc128bbcb41 req-638d44ce-851d-4c7c-926e-312b4eca0508 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:25.494830 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-477ed117-f33a-4187-9abd-adc128bbcb41 req-638d44ce-851d-4c7c-926e-312b4eca0508 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:25.505872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-477ed117-f33a-4187-9abd-adc128bbcb41 req-638d44ce-851d-4c7c-926e-312b4eca0508 service nova] Creating event network-changed:411b31d5-4a9b-44b2-8ce8-22d9cb075ef9 for instance 44e7298b-38f3-436b-98ed-e9157faaca5e on hv-cinder-maste Sep 30 09:50:25.511246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-477ed117-f33a-4187-9abd-adc128bbcb41 req-638d44ce-851d-4c7c-926e-312b4eca0508 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032618 Sep 30 09:50:25.512027 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 263/538] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:26.098682 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:26.099299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:26.106126 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:26.106619 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:26.107066 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:26.108131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:26.108583 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:26.109164 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:26.206791 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:26.215270 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9a313408-b441-4836-9530-1c1248975403 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1244 microversion: 2.1 time: 0.120820 Sep 30 09:50:26.215976 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 276/539] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:26 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1244 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:27.239645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:27.241060 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:27.255694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:27.255694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:27.255694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:27.255694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:27.256056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:27.256056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:27.324365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:27.332859 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-120ca5bd-8baf-490f-a175-6f020d56a146 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1292 microversion: 2.1 time: 0.098188 Sep 30 09:50:27.333421 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 264/540] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:27 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1292 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:27.527196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-99429585-c008-4348-b34b-524bb91bf8a8 req-31576366-e270-4217-875d-8caf13f0bf4b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "44e7298b-38f3-436b-98ed-e9157faaca5e", "tag": "411b31d5-4a9b-44b2-8ce8-22d9cb075ef9"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:27.537470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-99429585-c008-4348-b34b-524bb91bf8a8 req-31576366-e270-4217-875d-8caf13f0bf4b service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:27.537945 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-99429585-c008-4348-b34b-524bb91bf8a8 req-31576366-e270-4217-875d-8caf13f0bf4b service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:27.538419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-99429585-c008-4348-b34b-524bb91bf8a8 req-31576366-e270-4217-875d-8caf13f0bf4b service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:27.549878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-99429585-c008-4348-b34b-524bb91bf8a8 req-31576366-e270-4217-875d-8caf13f0bf4b service nova] Creating event network-changed:411b31d5-4a9b-44b2-8ce8-22d9cb075ef9 for instance 44e7298b-38f3-436b-98ed-e9157faaca5e on hv-cinder-maste Sep 30 09:50:27.554843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-99429585-c008-4348-b34b-524bb91bf8a8 req-31576366-e270-4217-875d-8caf13f0bf4b service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031699 Sep 30 09:50:27.555470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 277/541] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:28.356265 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:28.357096 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:28.363770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:28.364246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:28.364735 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:28.365424 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:28.365873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:28.366351 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:28.431437 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:28.439226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5bc94207-ce1e-4a43-ba7d-7dc278ffdf23 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1298 microversion: 2.1 time: 0.087252 Sep 30 09:50:28.439777 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 265/542] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:28 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1298 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:29.266866 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-431625843", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "networks": [{"uuid": "70073525-5499-4f0a-9017-eb0ecf8cf458"}]}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:29.321496 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] validate_networks() for [('70073525-5499-4f0a-9017-eb0ecf8cf458', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:50:29.462106 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:29.462837 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:29.469497 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:29.470164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:29.471143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:29.474435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:29.474435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:29.474435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:29.944275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:29.952192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-659ebb6c-4598-4e63-b75e-f829a9551be8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1365 microversion: 2.1 time: 0.494256 Sep 30 09:50:29.952754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 266/543] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:29 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1365 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:29.957922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-4794bce8-2acd-4e1f-9594-05c0ce83d197 req-874421d9-174a-4788-b481-e17d4441c264 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27d777c7-ef67-43f8-a47c-eafb863feb81", "tag": "3e38c79a-a565-45f6-a69a-e39595512b01"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:29.967464 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-4794bce8-2acd-4e1f-9594-05c0ce83d197 req-874421d9-174a-4788-b481-e17d4441c264 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:29.967941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-4794bce8-2acd-4e1f-9594-05c0ce83d197 req-874421d9-174a-4788-b481-e17d4441c264 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:29.968413 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-4794bce8-2acd-4e1f-9594-05c0ce83d197 req-874421d9-174a-4788-b481-e17d4441c264 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:29.979953 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-4794bce8-2acd-4e1f-9594-05c0ce83d197 req-874421d9-174a-4788-b481-e17d4441c264 service nova] Creating event network-changed:3e38c79a-a565-45f6-a69a-e39595512b01 for instance 27d777c7-ef67-43f8-a47c-eafb863feb81 on hv-cinder-maste Sep 30 09:50:29.984908 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-4794bce8-2acd-4e1f-9594-05c0ce83d197 req-874421d9-174a-4788-b481-e17d4441c264 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030854 Sep 30 09:50:29.985784 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 267/544] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:29.996555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:50:29.997115 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:50:29.997713 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:50:29.998123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:50:29.998527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:50:30.007213 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:30.007814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:30.008245 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:30.008963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:30.009413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:30.009832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:30.026617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Getting quotas for project b3db64185ae94cefa4efa6f3e03c2f7e. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:50:30.030548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Getting quotas for user 8f8db95b40ca4350b4f86ce9f1593194 and project b3db64185ae94cefa4efa6f3e03c2f7e. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:50:30.040086 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:30.042535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:50:30.044940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:50:30.177835 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e8bd78c8-74d1-4ed1-b464-de194f1949c5 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.915890 Sep 30 09:50:30.178699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 278/545] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:50:29 2022] POST /compute/v2.1/servers => generated 380 bytes in 918 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:50:30.189978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-fe7c9b19-82a9-4ed1-8d3a-81e7eec02743 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:30.190690 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-fe7c9b19-82a9-4ed1-8d3a-81e7eec02743 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:30.243762 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fe7c9b19-82a9-4ed1-8d3a-81e7eec02743 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:30.251068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-fe7c9b19-82a9-4ed1-8d3a-81e7eec02743 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1218 microversion: 2.1 time: 0.065278 Sep 30 09:50:30.251631 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 268/546] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:30 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1218 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:30.977958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:30.978563 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:30.985529 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:30.986022 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:30.987128 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:30.987815 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:30.988262 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:30.988680 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:31.115501 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:31.123392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c7cd142c-6ebf-4f89-9a34-7e5d4656cc46 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1365 microversion: 2.1 time: 0.150453 Sep 30 09:50:31.124013 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 279/547] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:30 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1365 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:31.274565 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:31.275197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:31.282536 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:31.283536 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:31.284092 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:31.285003 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:31.285462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:31.285859 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:31.354142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:31.361883 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-aaf229f5-a903-4f45-885f-e5c5ad12d732 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1222 microversion: 2.1 time: 0.091536 Sep 30 09:50:31.362443 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 269/548] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:31 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1222 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:32.145781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:32.146392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:32.153082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:32.153564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:32.153999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:32.154704 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:32.155163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:32.155583 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:32.281431 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:32.288973 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-793e40de-e8b2-4660-9811-af0c0619ce3f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1365 microversion: 2.1 time: 0.147032 Sep 30 09:50:32.289540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 280/549] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:32 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1365 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:32.384686 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:32.385298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:32.391640 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:32.392114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:32.392547 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:32.393231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:32.393680 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:32.394100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:32.455147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:32.462651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0c95f12a-634f-4b09-a7df-8e99b572b57c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1270 microversion: 2.1 time: 0.081868 Sep 30 09:50:32.463213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 270/550] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:32 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1270 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:33.311565 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:33.312174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:33.319147 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:33.319637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:33.320068 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:33.320766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:33.321219 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:33.321642 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:33.464663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:33.466967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:33.473347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-26995ae5-78cb-493d-a2ec-1871cffb7c4e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1365 microversion: 2.1 time: 0.165669 Sep 30 09:50:33.476213 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 281/551] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:33 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1365 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:33.491878 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:33.494355 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:33.502169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:33.503529 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:33.504117 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:33.505451 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:33.506086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:33.507372 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:33.663814 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:33.672101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-263b7a8e-db7b-4ae6-9e41-bdd14977f91a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.185391 Sep 30 09:50:33.672764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 271/552] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:33 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:34.271856 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-1a360153-0a91-4440-8146-698a55ebf3a3 req-770f4e24-8a1e-42f4-9701-c186032ecfe3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae248c85-ab02-421e-9593-35d3e6b2d91f", "tag": "a9cb82d6-80f6-4a57-8692-df8965508945"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:34.282717 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-1a360153-0a91-4440-8146-698a55ebf3a3 req-770f4e24-8a1e-42f4-9701-c186032ecfe3 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:34.283274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-1a360153-0a91-4440-8146-698a55ebf3a3 req-770f4e24-8a1e-42f4-9701-c186032ecfe3 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:34.283914 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-1a360153-0a91-4440-8146-698a55ebf3a3 req-770f4e24-8a1e-42f4-9701-c186032ecfe3 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:34.296567 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-1a360153-0a91-4440-8146-698a55ebf3a3 req-770f4e24-8a1e-42f4-9701-c186032ecfe3 service nova] Creating event network-changed:a9cb82d6-80f6-4a57-8692-df8965508945 for instance ae248c85-ab02-421e-9593-35d3e6b2d91f on hv-cinder-maste Sep 30 09:50:34.301259 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-1a360153-0a91-4440-8146-698a55ebf3a3 req-770f4e24-8a1e-42f4-9701-c186032ecfe3 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033791 Sep 30 09:50:34.301832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 282/553] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:34.495565 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:34.496493 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:34.498537 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:34.504335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:34.504816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:34.505247 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:34.505938 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:34.506385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:34.506858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:34.636343 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:34.638769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:34.646240 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0cf17ba3-2c63-4e52-8094-4c92a405263e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1365 microversion: 2.1 time: 0.154656 Sep 30 09:50:34.651159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 272/554] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:34 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1365 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:34.706553 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:34.708003 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:34.714251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:34.714762 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:34.715185 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:34.715871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:34.716314 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:34.716738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:34.848776 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:34.856357 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b16d264e-7f8c-4449-87ec-53c4d5d57cb7 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.154844 Sep 30 09:50:34.857633 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 283/555] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:34 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:35.615533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-8ffb7890-f6e2-4e29-8ac1-bb7c92eab8a1 req-f928d2ed-ea99-41fd-a81f-f6c66c0b2df2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "27d777c7-ef67-43f8-a47c-eafb863feb81", "name": "network-vif-plugged", "status": "completed", "tag": "3e38c79a-a565-45f6-a69a-e39595512b01"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:35.620645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=2 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:35.627925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-8ffb7890-f6e2-4e29-8ac1-bb7c92eab8a1 req-f928d2ed-ea99-41fd-a81f-f6c66c0b2df2 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:35.633108 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-8ffb7890-f6e2-4e29-8ac1-bb7c92eab8a1 req-f928d2ed-ea99-41fd-a81f-f6c66c0b2df2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:35.633108 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-8ffb7890-f6e2-4e29-8ac1-bb7c92eab8a1 req-f928d2ed-ea99-41fd-a81f-f6c66c0b2df2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:35.639877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-8ffb7890-f6e2-4e29-8ac1-bb7c92eab8a1 req-f928d2ed-ea99-41fd-a81f-f6c66c0b2df2 service nova] Creating event network-vif-plugged:3e38c79a-a565-45f6-a69a-e39595512b01 for instance 27d777c7-ef67-43f8-a47c-eafb863feb81 on hv-cinder-maste Sep 30 09:50:35.645537 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-8ffb7890-f6e2-4e29-8ac1-bb7c92eab8a1 req-f928d2ed-ea99-41fd-a81f-f6c66c0b2df2 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033670 Sep 30 09:50:35.646114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 273/556] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:35.668664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:35.671373 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:35.675219 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:35.675763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:35.676195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:35.676882 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:35.677338 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:35.677763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:35.807013 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:35.815178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bc7ada46-fc10-4582-97a0-ad206116b22f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1365 microversion: 2.1 time: 0.150352 Sep 30 09:50:35.815824 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 284/557] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:35 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1365 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:35.880624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:35.881344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:35.888695 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:35.889162 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:35.889581 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:35.890324 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:35.890728 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:35.891145 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:36.021931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:36.029397 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3c07fa56-9ee4-4665-9ac9-7623ece7378b tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.153570 Sep 30 09:50:36.029955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 274/558] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:35 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:36.838667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:36.839276 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:36.846304 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:36.846763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:36.847194 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:36.847873 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:36.848324 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:36.848743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:36.975276 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:36.982801 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-26174ce6-772e-44f7-8692-4e24d9da7250 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1365 microversion: 2.1 time: 0.148063 Sep 30 09:50:36.983367 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 285/559] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:36 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1365 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:37.050959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:37.051562 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:37.057780 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:37.058252 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:37.058713 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:37.059398 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:37.059839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:37.060250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:37.190054 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:37.197637 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-96a60c27-bd58-4859-af9e-c49ace87ac40 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.150592 Sep 30 09:50:37.198188 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 275/560] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:37 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:38.005086 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:38.005693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:38.012095 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:38.012579 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:38.013010 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:38.013699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:38.014150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:38.014574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:38.141177 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:38.148901 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f532a636-5a4c-43e0-bcde-b7756115800c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1365 microversion: 2.1 time: 0.147652 Sep 30 09:50:38.149468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 286/561] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:38 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1365 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:38.222773 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:38.223376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:38.229715 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:38.230190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:38.230647 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:38.231329 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:38.231775 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:38.232189 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:38.358571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:38.366342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-68946052-962b-4d25-8e05-08b0ea02c3cf tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.148188 Sep 30 09:50:38.366938 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 276/562] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:38 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:39.172047 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:39.172682 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:39.179722 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:39.180203 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:39.180638 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:39.181334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:39.181786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:39.182208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:39.314621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:39.322226 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-37947cc1-5311-40fe-aaf8-d18849a69d66 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1556 microversion: 2.1 time: 0.154048 Sep 30 09:50:39.322817 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 287/563] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:39 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1556 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:39.343799 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:39.344286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:39.351008 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:39.351481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:39.351909 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:39.352587 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:39.353028 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:39.353439 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:39.388996 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:39.389601 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:39.395443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:39.395928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:39.396361 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:39.397050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:39.397774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:39.397896 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:39.432884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-31dbf7a2-e956-4be5-be32-1ff841ef8fd8 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.092817 Sep 30 09:50:39.433471 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 277/564] 10.222.0.39 () {58 vars in 1275 bytes} [Fri Sep 30 09:50:39 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-interface => generated 285 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:39.657144 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:39.665698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8c6acb7e-d24c-4a96-b2c6-3f362e7889fd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.280347 Sep 30 09:50:39.678657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 288/565] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:39 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:40.517092 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=8 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:40.521505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-3acc05ac-c89f-42f5-9d4d-92a0dfea54cb req-bc6006d6-1bec-4038-b47f-ea789b9c6dda service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27d777c7-ef67-43f8-a47c-eafb863feb81", "tag": "3e38c79a-a565-45f6-a69a-e39595512b01"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:40.530844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3acc05ac-c89f-42f5-9d4d-92a0dfea54cb req-bc6006d6-1bec-4038-b47f-ea789b9c6dda service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:40.531323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3acc05ac-c89f-42f5-9d4d-92a0dfea54cb req-bc6006d6-1bec-4038-b47f-ea789b9c6dda service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:40.531798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3acc05ac-c89f-42f5-9d4d-92a0dfea54cb req-bc6006d6-1bec-4038-b47f-ea789b9c6dda service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:40.542845 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-3acc05ac-c89f-42f5-9d4d-92a0dfea54cb req-bc6006d6-1bec-4038-b47f-ea789b9c6dda service nova] Creating event network-changed:3e38c79a-a565-45f6-a69a-e39595512b01 for instance 27d777c7-ef67-43f8-a47c-eafb863feb81 on hv-cinder-maste Sep 30 09:50:40.549071 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-3acc05ac-c89f-42f5-9d4d-92a0dfea54cb req-bc6006d6-1bec-4038-b47f-ea789b9c6dda service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032175 Sep 30 09:50:40.549719 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 278/566] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:40.689258 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:40.691216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:40.696878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:40.697808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=12 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:40.700085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:40.700911 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:40.701548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:40.706810 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:40.706810 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:40.706810 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:40.852114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:40.855225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1be074e8-fc51-4458-bf95-54d775703884 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.169770 Sep 30 09:50:40.855804 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 289/567] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:40 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:50:41.738091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-42480188-14ab-424c-a915-393c82d9960b req-d1d85123-e71a-41b3-8e5c-077006e69742 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae248c85-ab02-421e-9593-35d3e6b2d91f", "name": "network-vif-plugged", "status": "completed", "tag": "a9cb82d6-80f6-4a57-8692-df8965508945"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:41.748093 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-42480188-14ab-424c-a915-393c82d9960b req-d1d85123-e71a-41b3-8e5c-077006e69742 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:41.748540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-42480188-14ab-424c-a915-393c82d9960b req-d1d85123-e71a-41b3-8e5c-077006e69742 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:41.748957 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-42480188-14ab-424c-a915-393c82d9960b req-d1d85123-e71a-41b3-8e5c-077006e69742 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:41.759907 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-42480188-14ab-424c-a915-393c82d9960b req-d1d85123-e71a-41b3-8e5c-077006e69742 service nova] Creating event network-vif-plugged:a9cb82d6-80f6-4a57-8692-df8965508945 for instance ae248c85-ab02-421e-9593-35d3e6b2d91f on hv-cinder-maste Sep 30 09:50:41.764783 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-42480188-14ab-424c-a915-393c82d9960b req-d1d85123-e71a-41b3-8e5c-077006e69742 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031072 Sep 30 09:50:41.765187 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 279/568] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:41.879885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:41.880436 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:41.886995 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:41.887723 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:41.888163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:41.888863 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:41.889317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:41.889741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:42.018453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:42.025999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-58e1e593-c41a-428f-b4bf-aadf0f9c9e12 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.150667 Sep 30 09:50:42.026557 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 290/569] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:41 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:43.049590 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:43.050295 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:43.056407 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:43.056877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:43.057301 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:43.057976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:43.058418 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:43.058860 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:43.184392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:43.192341 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6bc9c553-d118-49ab-851f-ef59fdb2c90d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1318 microversion: 2.1 time: 0.147511 Sep 30 09:50:43.192897 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 280/570] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:43 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1318 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:44.215611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:44.216227 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:44.223173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:44.223653 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:44.224086 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:44.224770 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:44.225224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:44.225646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:44.354464 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:50:44.362018 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bed4e20c-35be-4154-937b-8885a087321d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1507 microversion: 2.1 time: 0.150333 Sep 30 09:50:44.362581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 291/571] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:50:44 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1507 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:50:55.459873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=17 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:55.461273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=35 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:55.466512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1664535141"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:55.467067 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:55.473404 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:55.473877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:55.474335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:55.475736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:55.476244 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:55.476663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:55.608638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.network.security_group_api [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Removing security group a2f2441d-2469-4195-ba5b-d5e2f6ba3152 from port 4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 Sep 30 09:50:55.980266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0a79ab0a-ecc0-455a-aebd-fba9dcbc32ae tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "POST /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/action" status: 202 len: 0 microversion: 2.1 time: 0.522774 Sep 30 09:50:55.980962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 281/572] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:50:55 2022] POST /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/action => generated 0 bytes in 524 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:50:55.988673 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-8be4ee59-c44c-465a-9428-bacce26c8f74 req-de0bd926-f2e4-49c3-82ee-0339e5d1d67c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35cb36cf-5501-4cf6-8a17-c988d7a0e337", "tag": "4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:55.995631 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=28,nova_api:INSERT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:55.996306 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=57 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:56.002256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-8be4ee59-c44c-465a-9428-bacce26c8f74 req-de0bd926-f2e4-49c3-82ee-0339e5d1d67c service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:56.008983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-8be4ee59-c44c-465a-9428-bacce26c8f74 req-de0bd926-f2e4-49c3-82ee-0339e5d1d67c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:56.008983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-8be4ee59-c44c-465a-9428-bacce26c8f74 req-de0bd926-f2e4-49c3-82ee-0339e5d1d67c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:56.019299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-8be4ee59-c44c-465a-9428-bacce26c8f74 req-de0bd926-f2e4-49c3-82ee-0339e5d1d67c service nova] Creating event network-changed:4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 for instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 on hv-cinder-maste Sep 30 09:50:56.024466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-8be4ee59-c44c-465a-9428-bacce26c8f74 req-de0bd926-f2e4-49c3-82ee-0339e5d1d67c service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039967 Sep 30 09:50:56.025387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 292/573] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:50:57.457305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:57.457854 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:57.463733 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:57.464210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:57.464690 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:57.465369 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:57.465843 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:57.466219 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:57.670366 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lazy-loading 'services' on Instance uuid 35cb36cf-5501-4cf6-8a17-c988d7a0e337 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:50:57.898730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d138a6fa-3667-4df9-8d08-60f08f782af3 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "DELETE /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments/9392ad7b-d061-4ce1-b5ac-59db92be7d11" status: 202 len: 0 microversion: 2.1 time: 0.445693 Sep 30 09:50:57.899522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 282/574] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:50:57 2022] DELETE /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments/9392ad7b-d061-4ce1-b5ac-59db92be7d11 => generated 0 bytes in 447 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:50:57.908966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:57.916967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:57.919021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:57.919294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:57.919727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:57.920429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:57.920888 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:57.921307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:57.950610 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-20f8a4b8-bf42-42f4-ae8c-7a51f49a8db5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046626 Sep 30 09:50:57.951566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 293/575] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:50:57 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:58.041412 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-21e44fae-df76-4472-8b9c-6b2c841f25f6 req-5d9f2134-301e-48f3-abc5-ec1c91cebde9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35cb36cf-5501-4cf6-8a17-c988d7a0e337", "tag": "4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:50:58.051551 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-21e44fae-df76-4472-8b9c-6b2c841f25f6 req-5d9f2134-301e-48f3-abc5-ec1c91cebde9 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:58.052230 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-21e44fae-df76-4472-8b9c-6b2c841f25f6 req-5d9f2134-301e-48f3-abc5-ec1c91cebde9 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:58.052654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-21e44fae-df76-4472-8b9c-6b2c841f25f6 req-5d9f2134-301e-48f3-abc5-ec1c91cebde9 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:58.064239 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-21e44fae-df76-4472-8b9c-6b2c841f25f6 req-5d9f2134-301e-48f3-abc5-ec1c91cebde9 service nova] Creating event network-changed:4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 for instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 on hv-cinder-maste Sep 30 09:50:58.069567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-21e44fae-df76-4472-8b9c-6b2c841f25f6 req-5d9f2134-301e-48f3-abc5-ec1c91cebde9 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032164 Sep 30 09:50:58.070339 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 283/576] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:50:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:50:58.964148 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:50:58.966310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:50:58.973204 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:58.973698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:58.974134 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:58.975438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:50:58.975922 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:50:58.976371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:50:59.007783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ce013d9b-8750-40ab-8514-713a21562a03 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047659 Sep 30 09:50:59.008371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 294/577] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:50:58 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:00.021352 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:00.023656 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:00.029716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:00.030192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:00.030652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:00.031335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:00.031785 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:00.032201 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:00.060090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-67c9c9b0-c321-4ddf-914d-98d961716666 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043679 Sep 30 09:51:00.060624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 284/578] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:00 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 09:51:00.071808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:00.072309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:00.078535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:00.079064 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:00.079495 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:00.080180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:00.080628 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:00.081046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:00.203561 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] HTTP exception thrown: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 is not attached to volume 9392ad7b-d061-4ce1-b5ac-59db92be7d11 Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Request method failure captured: Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments/9392ad7b-d061-4ce1-b5ac-59db92be7d11 HTTP/1.1 Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrtTEYyp5jy13XWRhYfWDrVbAEjCl0i_o8MIshrw4TRHa5toAfYdMjn6fDiHb-bnnh8tVZ7KDYGxrp0Yoxr5DrGlKrzCluXGlTmPXchCcmmRk5xoHGPuci43JHmgwBh8sZSgEVZS7BmLfBHyHuJq8ZLeIrrO_3Lv6zQYG4reF9y50Ys Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 0be64f8f1d4b452e84a7bcd64f6100d1 Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-TestMinimumBasicScenario-604383834 Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:51:00.205906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/0be64f8f1d4b452e84a7bcd64f6100d1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0be64f8f1d4b452e84a7bcd64f6100d1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0be64f8f1d4b452e84a7bcd64f6100d1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_0be64f8f1d4b452e84a7bcd64f6100d1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-TestMinimumBasicScenario-604383834 Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 0be64f8f1d4b452e84a7bcd64f6100d1 Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-TestMinimumBasicScenario-604383834 Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-TestMinimumBasicScenario-604383834-project Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 4d46ec34caf74ea7b7c8f836c2f7ea6c Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-TestMinimumBasicScenario-604383834-project Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:51:00.207586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 is not attached to volume 9392ad7b-d061-4ce1-b5ac-59db92be7d11 Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'server_id': '35cb36cf-5501-4cf6-8a17-c988d7a0e337', 'id': '9392ad7b-d061-4ce1-b5ac-59db92be7d11'} Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 9392ad7b-d061-4ce1-b5ac-59db92be7d11. Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:51:00.208468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 is not attached to volume 9392ad7b-d061-4ce1-b5ac-59db92be7d11 Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 is not attached to volume 9392ad7b-d061-4ce1-b5ac-59db92be7d11 Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:00.209786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Returning 404 to user: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 is not attached to volume 9392ad7b-d061-4ce1-b5ac-59db92be7d11 {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:51:00.211392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-eae2fec9-a601-4289-95e5-a1bc86515c75 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "DELETE /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments/9392ad7b-d061-4ce1-b5ac-59db92be7d11" status: 404 len: 154 microversion: 2.1 time: 0.140255 Sep 30 09:51:00.211392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 295/579] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:51:00 2022] DELETE /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337/os-volume_attachments/9392ad7b-d061-4ce1-b5ac-59db92be7d11 => generated 154 bytes in 141 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:51:01.456383 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:01.456876 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:01.463176 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:01.463649 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:01.464079 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:01.464760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:01.465207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:01.465636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:01.489282 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:51:01.769917 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6c12f46a-5977-48c0-8e4c-aa14cebae69e tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "DELETE /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 204 len: 0 microversion: 2.1 time: 0.317293 Sep 30 09:51:01.770506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 285/580] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:51:01 2022] DELETE /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:51:01.778678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:01.779291 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:01.785863 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:01.786346 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:01.786839 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:01.787535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:01.787985 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:01.788408 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:01.901055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8e7db158-71fb-42e2-bb97-241e907786e0", "device": "/dev/sdb"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:01.903988 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:01.909755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:01.910230 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:01.910686 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:01.911382 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:01.911829 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:01.912246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:01.921761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:01.930343 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8f1158a2-c665-4ed8-8c1d-568bb0431bac tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1554 microversion: 2.1 time: 0.155491 Sep 30 09:51:01.930933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 296/581] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:01 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1554 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:02.953337 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:02.954024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:02.960468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:02.960947 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:02.961378 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:02.962064 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:02.962519 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:02.962969 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:03.088422 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:03.096623 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ab99fbd6-a1c4-4219-8b66-ea35b06e8dc5 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1554 microversion: 2.1 time: 0.147163 Sep 30 09:51:03.097182 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 297/582] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:02 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1554 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:51:04.121097 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:04.122316 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:04.129111 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:04.129600 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:04.130033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:04.130766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:04.131220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:04.131670 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:04.333878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:04.341573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4ea89055-fd3e-43f5-8146-b0fa77594ace tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1554 microversion: 2.1 time: 0.225033 Sep 30 09:51:04.342452 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 298/583] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:04 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1554 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:51:04.475327 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-917c8427-29b6-4ec2-a77c-c6f7876f554c tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "POST /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.578190 Sep 30 09:51:04.476266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 286/584] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:51:01 2022] POST /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f/os-volume_attachments => generated 194 bytes in 2580 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:51:04.608243 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-b982bf16-8add-4a00-88d8-62efdd8fb8f2 req-846312cb-63ba-4e6c-baad-a341a519b2b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35cb36cf-5501-4cf6-8a17-c988d7a0e337", "name": "network-vif-deleted", "tag": "4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:04.619225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-b982bf16-8add-4a00-88d8-62efdd8fb8f2 req-846312cb-63ba-4e6c-baad-a341a519b2b8 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:04.619712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-b982bf16-8add-4a00-88d8-62efdd8fb8f2 req-846312cb-63ba-4e6c-baad-a341a519b2b8 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:04.620134 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-b982bf16-8add-4a00-88d8-62efdd8fb8f2 req-846312cb-63ba-4e6c-baad-a341a519b2b8 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:04.631588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-b982bf16-8add-4a00-88d8-62efdd8fb8f2 req-846312cb-63ba-4e6c-baad-a341a519b2b8 service nova] Creating event network-vif-deleted:4c2d7372-a2b5-4fdb-9ea8-cb7f35b53fc3 for instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 on hv-cinder-maste Sep 30 09:51:04.636510 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-b982bf16-8add-4a00-88d8-62efdd8fb8f2 req-846312cb-63ba-4e6c-baad-a341a519b2b8 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032305 Sep 30 09:51:04.637091 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 299/585] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:51:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:05.365617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:05.366417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:05.373745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:05.374222 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:05.374685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:05.375396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:05.375819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:05.376275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:05.406940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:05.407411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:05.407835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:05.408271 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lazy-loading 'fault' on Instance uuid 35cb36cf-5501-4cf6-8a17-c988d7a0e337 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:05.452050 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d46ec34caf74ea7b7c8f836c2f7ea6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be64f8f1d4b452e84a7bcd64f6100d1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:05.459852 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-23887a66-1ee8-45da-909a-c073e96345a9 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 200 len: 1349 microversion: 2.1 time: 0.099844 Sep 30 09:51:05.461022 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 287/586] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:05 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 1349 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:06.484706 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:06.485405 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] [instance: 35cb36cf-5501-4cf6-8a17-c988d7a0e337] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:06.491578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:06.492073 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:06.492507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:06.493195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:06.493650 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:06.494073 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:06.507233 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] HTTP exception thrown: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 could not be found. Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Request method failure captured: Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 HTTP/1.1 Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrtTEYyp5jy13XWRhYfWDrVbAEjCl0i_o8MIshrw4TRHa5toAfYdMjn6fDiHb-bnnh8tVZ7KDYGxrp0Yoxr5DrGlKrzCluXGlTmPXchCcmmRk5xoHGPuci43JHmgwBh8sZSgEVZS7BmLfBHyHuJq8ZLeIrrO_3Lv6zQYG4reF9y50Ys Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 0be64f8f1d4b452e84a7bcd64f6100d1 Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-TestMinimumBasicScenario-604383834 Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:51:06.508970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/0be64f8f1d4b452e84a7bcd64f6100d1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0be64f8f1d4b452e84a7bcd64f6100d1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/0be64f8f1d4b452e84a7bcd64f6100d1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_0be64f8f1d4b452e84a7bcd64f6100d1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-TestMinimumBasicScenario-604383834 Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 0be64f8f1d4b452e84a7bcd64f6100d1 Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-TestMinimumBasicScenario-604383834 Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-TestMinimumBasicScenario-604383834-project Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 4d46ec34caf74ea7b7c8f836c2f7ea6c Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-TestMinimumBasicScenario-604383834-project Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 could not be found. Sep 30 09:51:06.510749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '35cb36cf-5501-4cf6-8a17-c988d7a0e337'} Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 could not be found. Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:51:06.511588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 could not be found. Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 could not be found. Sep 30 09:51:06.513053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:06.514513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Returning 404 to user: Instance 35cb36cf-5501-4cf6-8a17-c988d7a0e337 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:51:06.514513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-144e5337-8437-42e5-8cf5-8578e76403ce tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337" status: 404 len: 111 microversion: 2.1 time: 0.030535 Sep 30 09:51:06.514513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 300/587] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:06 2022] GET /compute/v2.1/servers/35cb36cf-5501-4cf6-8a17-c988d7a0e337 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:51:06.519076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7e7ad1a5-ba5c-43f9-9a37-b484373c4d19 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] Calling method '.version_select at 0x7f72c0b3d280>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:06.591322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7e7ad1a5-ba5c-43f9-9a37-b484373c4d19 tempest-TestMinimumBasicScenario-604383834 tempest-TestMinimumBasicScenario-604383834-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1896860384" status: 202 len: 0 microversion: 2.1 time: 0.075947 Sep 30 09:51:06.591900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 288/588] 10.222.0.39 () {58 vars in 1272 bytes} [Fri Sep 30 09:51:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1896860384 => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 30 09:51:13.259389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:13.259943 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:13.267482 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:13.268342 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:13.268870 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:13.269626 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:13.270245 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:13.270970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:13.361961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lazy-loading 'services' on Instance uuid 44e7298b-38f3-436b-98ed-e9157faaca5e {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:13.635396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7e715c20-7d32-4f48-8087-1167db185fcf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "DELETE /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments/f1609c14-b9aa-4f3c-956a-306e64f3ff52" status: 202 len: 0 microversion: 2.1 time: 0.380308 Sep 30 09:51:13.635957 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 301/589] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:51:13 2022] DELETE /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments/f1609c14-b9aa-4f3c-956a-306e64f3ff52 => generated 0 bytes in 382 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:51:13.644999 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:13.647188 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:13.654069 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:13.654551 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:13.655009 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:13.655699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:13.656212 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:13.656629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:13.689147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-36fca749-0ba2-40cb-956d-235a4d0dd7ee tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048498 Sep 30 09:51:13.689892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 289/590] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:13 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:14.713325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:14.716538 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:14.724348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:14.725028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:14.725621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:14.726541 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:14.727224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:14.727646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:14.765076 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e81d81f8-6c25-486f-b5eb-a90050baaf1c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060696 Sep 30 09:51:14.770262 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 302/591] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:14 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:15.778022 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:15.780199 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:15.786999 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:15.787490 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:15.787928 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:15.789204 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:15.789685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:15.790128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:15.822334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8b4735a4-b27e-461b-9e7c-666500a71b19 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048011 Sep 30 09:51:15.822681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 290/592] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:15 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:51:16.834409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:16.836567 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:16.843675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:16.844205 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:16.844665 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:16.845392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:16.845845 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:16.846261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:16.877491 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7545929a-a446-40ee-a78b-229670e1a7cf tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047125 Sep 30 09:51:16.878073 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 303/593] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:16 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:17.890042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:17.892191 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:17.898643 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:17.899075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:17.899500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:17.900183 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:17.900624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:17.901034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:17.930136 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e6d92ad2-3958-4dd7-bbcb-8eebea99339f tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043870 Sep 30 09:51:17.936070 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 291/594] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:17 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 09:51:18.018178 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:18.018827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:18.024933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:18.025436 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:18.025862 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:18.026561 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:18.027043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:18.027603 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:18.055160 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:51:18.409856 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e628026d-4b62-4fab-9ee8-b988c1c56ff7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "DELETE /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 204 len: 0 microversion: 2.1 time: 0.394982 Sep 30 09:51:18.410548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 304/595] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:51:18 2022] DELETE /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:51:18.419049 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:18.419717 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:18.426855 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:18.427347 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:18.427834 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:18.428560 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:18.428978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:18.429949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:18.566518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:18.574273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1fa0cb2a-3526-4dcd-870a-f6d28bffb05c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1690 microversion: 2.1 time: 0.159159 Sep 30 09:51:18.574865 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 292/596] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:18 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1690 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:19.598229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:19.598953 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:19.605552 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:19.606037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:19.606694 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:19.607729 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:19.608183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:19.608601 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:19.864822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:19.885040 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bc0a2df8-c4b1-4d0f-bf38-5465dd12a636 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1690 microversion: 2.1 time: 0.281656 Sep 30 09:51:19.886293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 305/597] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:19 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1690 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:20.909418 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:20.910385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:20.917969 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:20.919387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:20.920371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:20.921314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:20.921968 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:20.922492 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:21.154391 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:21.162327 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-03c890e8-9a3b-44d8-8349-ef422c4bdef0 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1690 microversion: 2.1 time: 0.256803 Sep 30 09:51:21.164119 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 293/598] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:20 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1690 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:22.189174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:22.190377 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:22.197519 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:22.198271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:22.198900 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:22.199796 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:22.200472 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:22.200900 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:22.423984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:22.454756 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fd080c97-fa7e-4a83-a9aa-490692e00570 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1690 microversion: 2.1 time: 0.266057 Sep 30 09:51:22.454871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 306/599] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:22 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1690 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:23.483738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:23.486382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:23.498412 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:23.502067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:23.506970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:23.508402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:23.509085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:23.512229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:23.579885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-6c13b90b-53df-4144-9cf6-8f579ff5960d req-ceffe0eb-7059-4e60-9b5e-99aefa2f6886 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "44e7298b-38f3-436b-98ed-e9157faaca5e", "name": "network-vif-deleted", "tag": "411b31d5-4a9b-44b2-8ce8-22d9cb075ef9"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:23.604834 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-6c13b90b-53df-4144-9cf6-8f579ff5960d req-ceffe0eb-7059-4e60-9b5e-99aefa2f6886 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:23.605709 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-6c13b90b-53df-4144-9cf6-8f579ff5960d req-ceffe0eb-7059-4e60-9b5e-99aefa2f6886 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:23.606314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-6c13b90b-53df-4144-9cf6-8f579ff5960d req-ceffe0eb-7059-4e60-9b5e-99aefa2f6886 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:23.631095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-6c13b90b-53df-4144-9cf6-8f579ff5960d req-ceffe0eb-7059-4e60-9b5e-99aefa2f6886 service nova] Creating event network-vif-deleted:411b31d5-4a9b-44b2-8ce8-22d9cb075ef9 for instance 44e7298b-38f3-436b-98ed-e9157faaca5e on hv-cinder-maste Sep 30 09:51:23.645588 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-6c13b90b-53df-4144-9cf6-8f579ff5960d req-ceffe0eb-7059-4e60-9b5e-99aefa2f6886 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.326016 Sep 30 09:51:23.646906 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 294/600] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:51:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:23.697498 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:23.724735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-62c3c9be-e41e-4540-b927-028f2c5446a8 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1331 microversion: 2.1 time: 0.239133 Sep 30 09:51:23.742211 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 307/601] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:23 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1331 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:24.780022 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:24.781328 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:24.791085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:24.791857 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:24.792454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:24.793533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:24.794141 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:24.794946 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:24.887632 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:24.895515 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-38faec0a-7355-401d-b9af-a86e6f696097 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1331 microversion: 2.1 time: 0.120790 Sep 30 09:51:24.896084 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 295/602] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:24 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1331 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:25.920144 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:25.920757 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:25.928394 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:25.928899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:25.929331 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:25.930115 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:25.930669 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:25.931082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:25.935265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:25.970782 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:25.971453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:25.971699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:25.972139 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lazy-loading 'fault' on Instance uuid 44e7298b-38f3-436b-98ed-e9157faaca5e {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:26.029322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:26.037015 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fa80a6d7-14bf-4f7a-93c5-8fb0a128acfe tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1336 microversion: 2.1 time: 0.121618 Sep 30 09:51:26.037585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 308/603] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:25 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1336 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:26.501972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:26.502564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:26.504738 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=39,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:26.510288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:26.510822 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:26.511530 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:26.512463 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:26.513335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:26.515335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:26.576495 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:26.588635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-fa4fa9eb-db78-4454-bd48-4d3b129b5848 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/ips" status: 200 len: 151 microversion: 2.1 time: 0.427567 Sep 30 09:51:26.596091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 296/604] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 09:51:26 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/ips => generated 151 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:27.062270 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:27.063584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:27.072464 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:27.073850 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:27.074302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:27.075037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:27.075663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:27.076093 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:27.117923 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:27.118784 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:27.119232 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:27.119664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lazy-loading 'fault' on Instance uuid 44e7298b-38f3-436b-98ed-e9157faaca5e {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:27.170081 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:27.181250 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f5c9e7d9-ead3-46ce-a5e8-00c7cc8c55c6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 200 len: 1336 microversion: 2.1 time: 0.120867 Sep 30 09:51:27.181881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 309/605] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:27 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 1336 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:28.208507 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:28.209134 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: 44e7298b-38f3-436b-98ed-e9157faaca5e] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:28.218506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:28.219262 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:28.219894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:28.220779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:28.221396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:28.221975 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:28.237610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] HTTP exception thrown: Instance 44e7298b-38f3-436b-98ed-e9157faaca5e could not be found. Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Request method failure captured: Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e HTTP/1.1 Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrTlXnTtrSqp1V_x1W1vnIRpWmkBjrFT8cNypZpVQr27Bt1ba58D9-pK8ZLi7HjxFCHn6nlNUOFaxuG8SOFGbJyOl9-u5Y9sbOG4I2nKOR06Qh9xhjqVXK1QVMfkONcO4rT4iNkpR9PqGmjOgNcaMeH-UkdqodqHrtoefVyVzzrL0A Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:51:28.239322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 7ad21da05fc246d489b0afa641f20216 Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 44e7298b-38f3-436b-98ed-e9157faaca5e could not be found. Sep 30 09:51:28.240981 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '44e7298b-38f3-436b-98ed-e9157faaca5e'} Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 44e7298b-38f3-436b-98ed-e9157faaca5e could not be found. Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:51:28.242534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 44e7298b-38f3-436b-98ed-e9157faaca5e could not be found. Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 44e7298b-38f3-436b-98ed-e9157faaca5e could not be found. Sep 30 09:51:28.244053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:28.246575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Returning 404 to user: Instance 44e7298b-38f3-436b-98ed-e9157faaca5e could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:51:28.247639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c5617071-422b-4776-a13a-60e4f22bd245 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e" status: 404 len: 111 microversion: 2.1 time: 0.047599 Sep 30 09:51:28.252252 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 297/606] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:28 2022] GET /compute/v2.1/servers/44e7298b-38f3-436b-98ed-e9157faaca5e => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:51:28.258855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0b822620-4992-4052-abb9-5685454d01d5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '.version_select at 0x7f72c099e040>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:28.380133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0b822620-4992-4052-abb9-5685454d01d5 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1268259575" status: 202 len: 0 microversion: 2.1 time: 0.126828 Sep 30 09:51:28.381366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 310/607] 10.222.0.39 () {58 vars in 1221 bytes} [Fri Sep 30 09:51:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1268259575 => generated 0 bytes in 129 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:51:28.420576 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d8cf6740-b7df-49d6-8e88-9812b15e1fde"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:28.423780 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:28.431167 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:28.431655 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:28.432087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:28.432785 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:28.433259 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:28.433647 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:29.555717 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-430b7787-e1d7-4b9a-bff3-abb49a9f9fee tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.138899 Sep 30 09:51:29.558708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 298/608] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:51:28 2022] POST /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 194 bytes in 1141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:31.089294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:31.090119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:31.092195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:31.100352 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:31.104991 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:31.105500 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:31.107800 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:31.107800 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:31.107800 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:31.240911 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lazy-loading 'services' on Instance uuid ae248c85-ab02-421e-9593-35d3e6b2d91f {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:31.309579 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f4b5787f-8985-433f-a013-adff912963f7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Action: 'create', calling method: .version_select at 0x7f72c0b85430>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1635713456", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJBQGpDQKlnq9rm+4BScyxmmvu5YxEd29/NTz5u34HCrtghrEm0FPoOCG8MgG5yI3SCVsDb2JpnHZDuKvB+uxgrdlCSpHJamlD33ZLEMXSTngtNZkc7OXmyd7MmWjeGvgg=="}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:31.328032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-f4b5787f-8985-433f-a013-adff912963f7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Getting quotas for project f0d0c96bbb304882b1f2459361551a93. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:51:31.332222 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-f4b5787f-8985-433f-a013-adff912963f7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Getting quotas for user c1df4fe7054840feac0a648ce82eea1a and project f0d0c96bbb304882b1f2459361551a93. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:51:31.426435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f4b5787f-8985-433f-a013-adff912963f7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.272125 Sep 30 09:51:31.427057 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 299/609] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:51:31 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:31.462892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7d872bb0-bf1e-4221-93d2-36f9ac858827 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "DELETE /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f/os-volume_attachments/8e7db158-71fb-42e2-bb97-241e907786e0" status: 202 len: 0 microversion: 2.1 time: 0.378749 Sep 30 09:51:31.470428 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 311/610] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:51:31 2022] DELETE /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f/os-volume_attachments/8e7db158-71fb-42e2-bb97-241e907786e0 => generated 0 bytes in 380 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:51:32.301404 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:32.302090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:32.309707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:32.311106 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:32.311613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:32.312588 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:32.313176 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:32.313674 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:32.342996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:51:32.705233 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0c612970-97e3-4fd1-991c-85501cf9a2fc tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "DELETE /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 204 len: 0 microversion: 2.1 time: 0.408126 Sep 30 09:51:32.709570 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 300/611] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:51:32 2022] DELETE /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:51:32.716430 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:32.717133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:32.723915 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:32.724625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:32.725188 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:32.725915 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:32.726366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:32.726814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:32.886929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:32.894826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-48402364-b126-4bdf-a72b-e3c685e9cada tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1508 microversion: 2.1 time: 0.183335 Sep 30 09:51:32.895464 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 312/612] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:32 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1508 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:33.918176 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:33.919007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:33.930191 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:33.930766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:33.931429 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:33.932354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:33.933793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:33.934736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:34.078543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:34.086437 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-62e65458-ec4b-4992-a0c0-b3a4e859b76c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1508 microversion: 2.1 time: 0.172807 Sep 30 09:51:34.087760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 301/613] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:33 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1508 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:35.109935 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:35.110446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:35.116737 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:35.117242 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:35.117674 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:35.118360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:35.118849 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:35.119288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:35.252400 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:35.260348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e111b750-cc63-4ca1-8311-8ba113db0c76 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1508 microversion: 2.1 time: 0.154354 Sep 30 09:51:35.261465 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 313/614] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:35 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1508 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:35.943764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-9b69b266-95f4-4a4c-83f7-f7e4ea8c84fe req-f5f01054-5272-4c20-8caf-353bfd5d8006 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d2701f34-7f9f-49cb-a2c3-9f84c1a96e31", "name": "network-vif-deleted", "tag": "d3f7eb0a-905a-4efa-b08a-3fca026724aa"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:35.955181 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-9b69b266-95f4-4a4c-83f7-f7e4ea8c84fe req-f5f01054-5272-4c20-8caf-353bfd5d8006 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:35.960034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-9b69b266-95f4-4a4c-83f7-f7e4ea8c84fe req-f5f01054-5272-4c20-8caf-353bfd5d8006 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:35.960034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-9b69b266-95f4-4a4c-83f7-f7e4ea8c84fe req-f5f01054-5272-4c20-8caf-353bfd5d8006 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:35.972013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-9b69b266-95f4-4a4c-83f7-f7e4ea8c84fe req-f5f01054-5272-4c20-8caf-353bfd5d8006 service nova] Creating event network-vif-deleted:d3f7eb0a-905a-4efa-b08a-3fca026724aa for instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 on hv-cinder-maste Sep 30 09:51:35.977434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-9b69b266-95f4-4a4c-83f7-f7e4ea8c84fe req-f5f01054-5272-4c20-8caf-353bfd5d8006 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037488 Sep 30 09:51:35.978310 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 302/615] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:51:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:36.284324 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:36.284930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:36.292741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:36.293241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:36.293682 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:36.294392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:36.294880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:36.295308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:36.366794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:36.375592 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-19899c92-0e06-4bf9-bab4-dfc80ba2aa5b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1293 microversion: 2.1 time: 0.095224 Sep 30 09:51:36.376158 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 314/616] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:36 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1293 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:37.403959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:37.404657 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:37.411383 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:37.411865 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:37.412295 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:37.412992 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:37.413436 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:37.413867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:37.448817 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:37.449306 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:37.449736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:37.450184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lazy-loading 'fault' on Instance uuid d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:37.494072 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:37.502004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-db5e9617-fc90-4c0f-a7ec-817338902c86 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 200 len: 1312 microversion: 2.1 time: 0.102860 Sep 30 09:51:37.502569 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 303/617] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:37 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 1312 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:38.214649 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d8cf6740-b7df-49d6-8e88-9812b15e1fde"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:38.217063 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:38.223767 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.224843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.225352 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.226059 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.226515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.226965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.309738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Invalid volume: volume d8cf6740-b7df-49d6-8e88-9812b15e1fde is already attached to instances: 27d777c7-ef67-43f8-a47c-eafb863feb81 Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: POST /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments HTTP/1.1 Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 74 Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:51:38.311447 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {"volumeAttachment": {"volumeId": "d8cf6740-b7df-49d6-8e88-9812b15e1fde"}} Sep 30 09:51:38.313196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Invalid volume: volume d8cf6740-b7df-49d6-8e88-9812b15e1fde is already attached to instances: 27d777c7-ef67-43f8-a47c-eafb863feb81 Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'server_id': '27d777c7-ef67-43f8-a47c-eafb863feb81', 'body': {'volumeAttachment': {'volumeId': 'd8cf6740-b7df-49d6-8e88-9812b15e1fde'}}} Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume d8cf6740-b7df-49d6-8e88-9812b15e1fde is already attached to instances: 27d777c7-ef67-43f8-a47c-eafb863feb81 Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.314136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume d8cf6740-b7df-49d6-8e88-9812b15e1fde is already attached to instances: 27d777c7-ef67-43f8-a47c-eafb863feb81 Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:38.315672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:38.317218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:38.317218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:51:38.317218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:51:38.317218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume d8cf6740-b7df-49d6-8e88-9812b15e1fde is already attached to instances: 27d777c7-ef67-43f8-a47c-eafb863feb81 Sep 30 09:51:38.317218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.317218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 400 to user: Invalid volume: volume d8cf6740-b7df-49d6-8e88-9812b15e1fde is already attached to instances: 27d777c7-ef67-43f8-a47c-eafb863feb81 {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:51:38.317218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7dcd52c7-797b-40ba-bc1c-5c2f734cf25b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.103052 Sep 30 09:51:38.317218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 315/618] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:51:38 2022] POST /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 174 bytes in 104 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 30 09:51:38.377663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:38.378152 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:38.384013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.384982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.385437 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.386089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.386536 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.386975 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.469875 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lazy-loading 'services' on Instance uuid 27d777c7-ef67-43f8-a47c-eafb863feb81 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:38.529705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:38.530307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: d2701f34-7f9f-49cb-a2c3-9f84c1a96e31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:38.536334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.538951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.538951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.538951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.538951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.539338 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.553397 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] HTTP exception thrown: Instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 could not be found. Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Request method failure captured: Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 HTTP/1.1 Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrTlXnTtrSqp1V_x1W1vnIRpWmkBjrFT8cNypZpVQr27Bt1ba58D9-pK8ZLi7HjxFCHn6nlNUOFaxuG8SOFGbJyOl9-u5Y9sbOG4I2nKOR06Qh9xhjqVXK1QVMfkONcO4rT4iNkpR9PqGmjOgNcaMeH-UkdqodqHrtoefVyVzzrL0A Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:51:38.555184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 7ad21da05fc246d489b0afa641f20216 Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 could not be found. Sep 30 09:51:38.557001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': 'd2701f34-7f9f-49cb-a2c3-9f84c1a96e31'} Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 could not be found. Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:51:38.557977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 could not be found. Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 could not be found. Sep 30 09:51:38.559775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:38.561374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Returning 404 to user: Instance d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:51:38.561374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4ef56afb-6871-4562-ad4f-e2c684ef616b tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31" status: 404 len: 111 microversion: 2.1 time: 0.031489 Sep 30 09:51:38.561374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 316/619] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:38 2022] GET /compute/v2.1/servers/d2701f34-7f9f-49cb-a2c3-9f84c1a96e31 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 30 09:51:38.565594 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:38.566082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:38.572409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.573417 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.573417 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.574125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.574576 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.575020 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.603942 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:51:38.699347 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-91e9622d-6695-4fb0-8c63-455f5562118a tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments/d8cf6740-b7df-49d6-8e88-9812b15e1fde" status: 202 len: 0 microversion: 2.1 time: 0.325466 Sep 30 09:51:38.699913 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 304/620] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:51:38 2022] DELETE /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments/d8cf6740-b7df-49d6-8e88-9812b15e1fde => generated 0 bytes in 327 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:51:38.708690 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:38.711468 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:38.719965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.720607 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.721051 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.721748 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.722199 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.722682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.755455 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-08c04b08-ffdb-4666-b19d-93c404a0621f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050731 Sep 30 09:51:38.756020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 305/621] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:38 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:51:38.876601 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9a7a8781-c68b-4721-97f9-9f892ab0f5ad tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "DELETE /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 204 len: 0 microversion: 2.1 time: 0.314553 Sep 30 09:51:38.877208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 317/622] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:51:38 2022] DELETE /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 0 bytes in 316 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:51:38.885904 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:38.886514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:38.893293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.893875 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.894306 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.895033 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.895485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.895906 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.979225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:38.979963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:38.986237 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.986744 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.987173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:38.987864 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:38.988315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:38.988735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:39.017057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:51:39.036052 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:39.044144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8e12c3ea-173b-4a9f-8b90-0d254a226af6 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.162728 Sep 30 09:51:39.045086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 306/623] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:38 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:39.285094 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f1deb25b-0bf0-491c-ba12-098aca1c4a05 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "DELETE /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 204 len: 0 microversion: 2.1 time: 0.312425 Sep 30 09:51:39.285684 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 318/624] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:51:38 2022] DELETE /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:51:39.294531 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:39.295169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:39.301463 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:39.301945 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:39.302373 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:39.303146 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:39.303594 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:39.304010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:39.435481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:39.443305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-094a11a9-880d-414b-97ba-0eeb8e7f3ff2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1513 microversion: 2.1 time: 0.153022 Sep 30 09:51:39.447489 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 307/625] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:39 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1513 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:51:39.769188 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:39.771483 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:39.778001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:39.778622 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:39.779119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:39.779814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:39.780458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:39.780981 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:39.810475 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-51eaad83-0c3f-4c3c-8301-bbd20d0a4591 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045576 Sep 30 09:51:39.811056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 319/626] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:39 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:40.067968 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:40.068567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:40.075028 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:40.075507 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:40.075927 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:40.076667 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:40.077122 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:40.077612 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:40.214685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:40.222352 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-116d531c-6347-49a2-ade9-3f091455735e tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.158218 Sep 30 09:51:40.222967 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 308/627] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:40 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:40.465758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:40.466544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:40.473345 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:40.473825 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:40.474274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:40.475007 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:40.475455 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:40.475872 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:40.609819 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:40.617571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9b910967-22af-4f9f-a52f-ea26589d9081 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1513 microversion: 2.1 time: 0.156254 Sep 30 09:51:40.618262 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 320/628] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:40 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1513 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:40.823257 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:40.825574 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:40.831887 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:40.832355 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:40.832778 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:40.833822 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:40.834270 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:40.834705 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:40.865148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5705a760-0722-44d3-827c-c845e4338d5e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045922 Sep 30 09:51:40.865720 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 309/629] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:40 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:51:41.245368 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:41.246259 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:41.252775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:41.253269 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:41.253790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:41.254615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:41.255087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:41.255514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:41.387176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:41.390808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:41.395451 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ddd837c4-ce7a-47e0-908c-9a3914dbaf88 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.154095 Sep 30 09:51:41.396047 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 321/630] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:41 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:41.642152 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:41.642767 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:41.644782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=4 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:41.649434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:41.649848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:41.650273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:41.650977 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:41.651423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:41.651842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:41.782491 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:41.790417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2e07dbf5-a939-4920-a773-7e2942e2c0d8 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1513 microversion: 2.1 time: 0.152113 Sep 30 09:51:41.791019 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 310/631] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:41 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1513 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:41.878380 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:41.880898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:41.889383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:41.889879 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:41.890319 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:41.891062 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:41.891519 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:41.891946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:41.940065 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fa68797a-9207-4a98-9251-06d1a3644d00 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051299 Sep 30 09:51:41.940065 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 322/632] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:41 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:42.419280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:42.420044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:42.426575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:42.427079 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:42.427506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:42.428179 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:42.428622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:42.429037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:42.443957 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:42.561726 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:42.569778 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2f47e510-d48e-4081-a4db-99c777396641 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.155297 Sep 30 09:51:42.570405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 311/633] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:42 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:42.815051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:42.815659 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:42.822006 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:42.822481 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:42.822936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:42.823627 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:42.824078 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:42.824494 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:42.938477 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:42.940887 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:42.946997 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:42.947472 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:42.947948 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:42.948634 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:42.949077 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:42.949520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:42.957380 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:42.965542 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c0903df1-854b-474c-b3d1-a560277ec260 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1513 microversion: 2.1 time: 0.154630 Sep 30 09:51:42.966186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 323/634] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:42 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1513 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:42.978272 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b109ffbf-8b1c-406b-8dcf-a6ac146c023b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043810 Sep 30 09:51:42.979682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 312/635] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:42 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:51:43.594367 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:43.594914 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:43.600992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:43.604582 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:43.604582 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:43.604582 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:43.604582 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:43.605017 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:43.746342 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:43.754381 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1c421ff1-88a7-4ed6-b770-109f0e540dd7 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1521 microversion: 2.1 time: 0.163985 Sep 30 09:51:43.759280 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 324/636] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:43 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:43.997104 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:43.997796 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:43.998167 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:44.003169 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:44.007748 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.008474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.009257 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.010173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.010879 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.011501 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.013549 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.018479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.018479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.018479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.018479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.018987 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.067083 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2f2c24dc-9cbe-4c88-ac07-c8c74f7fca29 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079158 Sep 30 09:51:44.071119 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 325/637] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:43 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:44.217783 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:44.227030 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f2245f41-c3b2-49de-8b81-fdf253b45944 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1513 microversion: 2.1 time: 0.236900 Sep 30 09:51:44.232862 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 313/638] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:43 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1513 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:44.319334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-e922f4f5-591b-494a-8351-47ca7f93772e req-d0d7d794-d3ae-4d39-81e0-cb6cf9f4079f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd8e4eb8-1261-4abf-82ec-e1d8afda21ec", "name": "network-vif-deleted", "tag": "45105b4f-28b5-44d6-bdd3-42f2c0b33de7"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:44.328999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-e922f4f5-591b-494a-8351-47ca7f93772e req-d0d7d794-d3ae-4d39-81e0-cb6cf9f4079f service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.329498 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-e922f4f5-591b-494a-8351-47ca7f93772e req-d0d7d794-d3ae-4d39-81e0-cb6cf9f4079f service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.329924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-e922f4f5-591b-494a-8351-47ca7f93772e req-d0d7d794-d3ae-4d39-81e0-cb6cf9f4079f service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.342367 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-e922f4f5-591b-494a-8351-47ca7f93772e req-d0d7d794-d3ae-4d39-81e0-cb6cf9f4079f service nova] Creating event network-vif-deleted:45105b4f-28b5-44d6-bdd3-42f2c0b33de7 for instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec on hv-cinder-maste Sep 30 09:51:44.347009 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-e922f4f5-591b-494a-8351-47ca7f93772e req-d0d7d794-d3ae-4d39-81e0-cb6cf9f4079f service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031296 Sep 30 09:51:44.347840 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 326/639] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:51:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:44.503765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-04b32af3-7ef6-4cd2-a7fb-db05837561ac req-bad9ea25-6af2-4cb0-a97b-15aa697f0589 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae248c85-ab02-421e-9593-35d3e6b2d91f", "name": "network-vif-deleted", "tag": "a9cb82d6-80f6-4a57-8692-df8965508945"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:44.513843 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-04b32af3-7ef6-4cd2-a7fb-db05837561ac req-bad9ea25-6af2-4cb0-a97b-15aa697f0589 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.514310 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-04b32af3-7ef6-4cd2-a7fb-db05837561ac req-bad9ea25-6af2-4cb0-a97b-15aa697f0589 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.514764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-04b32af3-7ef6-4cd2-a7fb-db05837561ac req-bad9ea25-6af2-4cb0-a97b-15aa697f0589 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.526048 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-04b32af3-7ef6-4cd2-a7fb-db05837561ac req-bad9ea25-6af2-4cb0-a97b-15aa697f0589 service nova] Creating event network-vif-deleted:a9cb82d6-80f6-4a57-8692-df8965508945 for instance ae248c85-ab02-421e-9593-35d3e6b2d91f on hv-cinder-maste Sep 30 09:51:44.532452 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-04b32af3-7ef6-4cd2-a7fb-db05837561ac req-bad9ea25-6af2-4cb0-a97b-15aa697f0589 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090415 Sep 30 09:51:44.533065 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 314/640] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:51:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:44.782816 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:44.783861 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:44.790195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.795719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.795719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.795719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.795719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.796264 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.825625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:44.826295 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:44.826771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:44.827312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lazy-loading 'fault' on Instance uuid fd8e4eb8-1261-4abf-82ec-e1d8afda21ec {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:44.872667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ad21da05fc246d489b0afa641f20216', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1c0948ddb014e6f93d6069bb5203890', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:44.881293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b7507089-3c17-4868-a8d7-b0daf66698f3 tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 200 len: 1312 microversion: 2.1 time: 0.103512 Sep 30 09:51:44.881926 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 327/641] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:44 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 1312 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:45.080397 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:45.082630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:45.088743 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.089256 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.089714 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.090419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.090892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.091315 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.122251 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3fdd70f6-b46b-4668-a699-a5bbad3dd97e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046566 Sep 30 09:51:45.122835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 315/642] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:51:45 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 09:51:45.185187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:45.185822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:45.191696 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.192180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.192611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.193340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.193794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.194210 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.218746 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:51:45.251909 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:45.252848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:45.262844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.263507 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.264028 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.264769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.265184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.265658 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.297250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.297722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.298146 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.298889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lazy-loading 'fault' on Instance uuid ae248c85-ab02-421e-9593-35d3e6b2d91f {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:45.343520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:45.351146 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-548fca46-3213-492d-b758-ef584498247d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1308 microversion: 2.1 time: 0.103314 Sep 30 09:51:45.351707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 316/643] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:45 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1308 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:51:45.550674 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-939024e2-753e-4e04-bf89-41c180e6861e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 204 len: 0 microversion: 2.1 time: 0.369678 Sep 30 09:51:45.551224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 328/644] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:51:45 2022] DELETE /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:51:45.560032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:45.560689 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:45.566719 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.567206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.567638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.568328 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.568783 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.569206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.695638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:45.703935 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e25f1f34-93fc-487c-8270-76f3aca75990 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1681 microversion: 2.1 time: 0.148382 Sep 30 09:51:45.704510 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 317/645] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:45 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1681 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:45.905895 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:45.906620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] [instance: fd8e4eb8-1261-4abf-82ec-e1d8afda21ec] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:45.912951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.913432 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.913865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.914554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:45.915031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:45.915461 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:45.928914 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] HTTP exception thrown: Instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec could not be found. Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Request method failure captured: Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec HTTP/1.1 Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrTlXnTtrSqp1V_x1W1vnIRpWmkBjrFT8cNypZpVQr27Bt1ba58D9-pK8ZLi7HjxFCHn6nlNUOFaxuG8SOFGbJyOl9-u5Y9sbOG4I2nKOR06Qh9xhjqVXK1QVMfkONcO4rT4iNkpR9PqGmjOgNcaMeH-UkdqodqHrtoefVyVzzrL0A Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:51:45.930526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f1c0948ddb014e6f93d6069bb5203890"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: f1c0948ddb014e6f93d6069bb5203890 Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-209097963 Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 7ad21da05fc246d489b0afa641f20216 Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-ServerRescueNegativeTestJSON-209097963-project Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec could not be found. Sep 30 09:51:45.932127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': 'fd8e4eb8-1261-4abf-82ec-e1d8afda21ec'} Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec could not be found. Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:51:45.933664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec could not be found. Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec could not be found. Sep 30 09:51:45.935191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:45.936511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] Returning 404 to user: Instance fd8e4eb8-1261-4abf-82ec-e1d8afda21ec could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:51:45.936511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-53bb69fd-b483-4a55-b847-bc5cc5556f7c tempest-ServerRescueNegativeTestJSON-209097963 tempest-ServerRescueNegativeTestJSON-209097963-project] 10.222.0.39 "GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec" status: 404 len: 111 microversion: 2.1 time: 0.030313 Sep 30 09:51:45.936511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 329/646] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:45 2022] GET /compute/v2.1/servers/fd8e4eb8-1261-4abf-82ec-e1d8afda21ec => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:51:46.375142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:46.375708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:46.382195 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:46.382697 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:46.383154 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:46.383838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:46.384281 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:46.384693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:46.415081 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:46.415552 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:46.415972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:46.423348 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lazy-loading 'fault' on Instance uuid ae248c85-ab02-421e-9593-35d3e6b2d91f {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:46.457680 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:46.465708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5d6d9890-80c6-4fec-826b-185eeacd1495 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1308 microversion: 2.1 time: 0.094800 Sep 30 09:51:46.466496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 318/647] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:46 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1308 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:46.726990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:46.727627 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:46.733602 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:46.734087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:46.734519 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:46.735237 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:46.735690 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:46.736112 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:46.867423 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:46.875910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3744eb61-0a2f-437e-a53b-9c2e0bf144e6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1681 microversion: 2.1 time: 0.152976 Sep 30 09:51:46.876473 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 330/648] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:46 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1681 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:47.489317 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:47.490156 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:47.498539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:47.499563 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:47.500147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:47.501044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:47.501802 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:47.502423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:47.537984 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:47.539233 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:47.539816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:47.540417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lazy-loading 'fault' on Instance uuid ae248c85-ab02-421e-9593-35d3e6b2d91f {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:47.623730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:47.631777 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-72e5c1d5-e1a1-4645-b835-c9e045ba7980 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 200 len: 1308 microversion: 2.1 time: 0.146366 Sep 30 09:51:47.632963 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 319/649] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:47 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 1308 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:47.899585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:47.900317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:47.910687 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:47.911615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:47.912607 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:47.914062 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:47.915591 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:47.916424 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:48.170012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:48.181631 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9ba02348-d660-4bf6-9e18-ea24b3418387 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1681 microversion: 2.1 time: 0.285842 Sep 30 09:51:48.184990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 331/650] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:47 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1681 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:48.659628 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:48.659628 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: ae248c85-ab02-421e-9593-35d3e6b2d91f] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:48.663422 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-806c6c21-1633-4b71-9af9-020e68b3b2c6 req-65f5e17f-f6fd-4c39-8d39-6d99d282531d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "27d777c7-ef67-43f8-a47c-eafb863feb81", "name": "network-vif-deleted", "tag": "3e38c79a-a565-45f6-a69a-e39595512b01"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:48.675103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-806c6c21-1633-4b71-9af9-020e68b3b2c6 req-65f5e17f-f6fd-4c39-8d39-6d99d282531d service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:48.675580 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-806c6c21-1633-4b71-9af9-020e68b3b2c6 req-65f5e17f-f6fd-4c39-8d39-6d99d282531d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:48.676004 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-806c6c21-1633-4b71-9af9-020e68b3b2c6 req-65f5e17f-f6fd-4c39-8d39-6d99d282531d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:48.679803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:48.680290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:48.680718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:48.681415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:48.681861 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:48.683440 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:48.696834 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-806c6c21-1633-4b71-9af9-020e68b3b2c6 req-65f5e17f-f6fd-4c39-8d39-6d99d282531d service nova] Creating event network-vif-deleted:3e38c79a-a565-45f6-a69a-e39595512b01 for instance 27d777c7-ef67-43f8-a47c-eafb863feb81 on hv-cinder-maste Sep 30 09:51:48.703579 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-806c6c21-1633-4b71-9af9-020e68b3b2c6 req-65f5e17f-f6fd-4c39-8d39-6d99d282531d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043789 Sep 30 09:51:48.704652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 332/651] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:51:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:48.715068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] HTTP exception thrown: Instance ae248c85-ab02-421e-9593-35d3e6b2d91f could not be found. Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Request method failure captured: Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f HTTP/1.1 Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrvjxVxMeUAW3mNXYZq8IAl8AeycXWzOw699KjdLi7tX7ZycT1K2C1FqjzLqyayOYru21geWzNDOO49rLGiDgJwJ0tmeJQUIbqR0Fwx_tsx2csk4xGdb7WyvnG7HiwMWWCzWW-CuV3Qp3OIgIrmkFE01ONfi0ApYWQnu7ADC4yI8pho Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: b3db64185ae94cefa4efa6f3e03c2f7e Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1827020212 Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:51:48.716918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b3db64185ae94cefa4efa6f3e03c2f7e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b3db64185ae94cefa4efa6f3e03c2f7e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b3db64185ae94cefa4efa6f3e03c2f7e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b3db64185ae94cefa4efa6f3e03c2f7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-VolumesSnapshotTestJSON-1827020212 Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: b3db64185ae94cefa4efa6f3e03c2f7e Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1827020212 Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-VolumesSnapshotTestJSON-1827020212-project Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 8f8db95b40ca4350b4f86ce9f1593194 Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-VolumesSnapshotTestJSON-1827020212-project Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance ae248c85-ab02-421e-9593-35d3e6b2d91f could not be found. Sep 30 09:51:48.718832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': 'ae248c85-ab02-421e-9593-35d3e6b2d91f'} Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ae248c85-ab02-421e-9593-35d3e6b2d91f could not be found. Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:51:48.719770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ae248c85-ab02-421e-9593-35d3e6b2d91f could not be found. Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ae248c85-ab02-421e-9593-35d3e6b2d91f could not be found. Sep 30 09:51:48.721289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:51:48.722784 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Returning 404 to user: Instance ae248c85-ab02-421e-9593-35d3e6b2d91f could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:51:48.728513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5c1ebca7-dfb4-49a2-b5b5-a0aa3b887090 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f" status: 404 len: 111 microversion: 2.1 time: 0.073600 Sep 30 09:51:48.728513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 320/652] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:48 2022] GET /compute/v2.1/servers/ae248c85-ab02-421e-9593-35d3e6b2d91f => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:51:49.206401 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:49.207124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:49.215324 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:49.216132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:49.216823 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:49.217789 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:49.218435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:49.218894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:49.259094 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:49.259735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:49.260014 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:49.260478 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lazy-loading 'fault' on Instance uuid 27d777c7-ef67-43f8-a47c-eafb863feb81 {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:49.318646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:49.328832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-41ae7ab7-1c35-471f-b104-ffd7eed920b5 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1330 microversion: 2.1 time: 0.127024 Sep 30 09:51:49.329415 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 333/653] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:49 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1330 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:50.351403 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:50.352010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:50.358978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:50.359465 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:50.359902 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:50.360604 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:50.361055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:50.361486 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:50.393186 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:50.393684 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:50.394120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:50.394572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lazy-loading 'fault' on Instance uuid 27d777c7-ef67-43f8-a47c-eafb863feb81 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:50.669587 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:50.677288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-869f6592-0099-470e-afcb-7640a434b472 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 200 len: 1330 microversion: 2.1 time: 0.329779 Sep 30 09:51:50.677852 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 321/654] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:50 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 1330 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:51.699438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:51.700055 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:51.706521 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:51.716200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:51.716200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:51.716200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:51.716200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:51.716774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:51.723850 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 HTTP/1.1 Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:51:51.725497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:51:51.726928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '27d777c7-ef67-43f8-a47c-eafb863feb81'} Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:51:51.727961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:51:51.730071 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:51:51.731625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:51:51.731625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-36b4f76b-f5f1-4ff1-899c-4f221bbf5c42 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 404 len: 111 microversion: 2.1 time: 0.031383 Sep 30 09:51:51.731625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 334/655] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:51 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:51:51.736329 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e812ab7e-71cf-47f7-88ef-299c0eca28bb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '.version_select at 0x7f72c0b244c0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:51.809606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e812ab7e-71cf-47f7-88ef-299c0eca28bb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-91522501" status: 202 len: 0 microversion: 2.1 time: 0.076953 Sep 30 09:51:51.810183 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 322/656] 10.222.0.39 () {58 vars in 1215 bytes} [Fri Sep 30 09:51:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-91522501 => generated 0 bytes in 78 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:51:52.596927 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1b22db29-1765-443d-af0e-f450a7fa9385 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: .version_select at 0x7f72c09c6550>, body: {"keypair": {"name": "tempest-keypair-478929732", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLtkAw/Ort8e1SM+1bjhIWgFE6wb5l+XBZK/4w/FE+cGeqI8ZtkGcBw4qQkciFDx7wTaUlu9ayz+qRU0oGhuObfNafJ/11cuRMnNJzy2IXLQZGe6VraGEFFBaIlp2k0Zuw=="}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:52.613380 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-1b22db29-1765-443d-af0e-f450a7fa9385 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for project 86e6b0501cb94028be07bddd2182638b. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:51:52.617641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-1b22db29-1765-443d-af0e-f450a7fa9385 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for user 8c4ced79c3ff4842b30b472acb555cd5 and project 86e6b0501cb94028be07bddd2182638b. Resources: {'key_pairs'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:51:52.704330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1b22db29-1765-443d-af0e-f450a7fa9385 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.111151 Sep 30 09:51:52.710106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 335/657] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:51:52 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:51:53.094829 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1828754714", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1635713456", "security_groups": [{"name": "tempest-secgroup-smoke-844905994"}], "block_device_mapping_v2": [{"uuid": "7bf1001d-9e49-498a-95db-ad274a5eff37", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3408b8b1-1a84-4fa0-8af6-da7419c2525d"}]}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:53.228526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] validate_networks() for [('3408b8b1-1a84-4fa0-8af6-da7419c2525d', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:51:53.770012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:51:53.770012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:51:53.771140 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:51:53.773084 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:51:53.773754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:51:53.802436 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:53.804152 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:53.804651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:53.805427 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:53.805874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:53.806324 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:53.824906 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Getting quotas for project f0d0c96bbb304882b1f2459361551a93. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:51:53.829114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Getting quotas for user c1df4fe7054840feac0a648ce82eea1a and project f0d0c96bbb304882b1f2459361551a93. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:51:53.900184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:51:53.903384 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='7bf1001d-9e49-498a-95db-ad274a5eff37',volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:51:54.356781 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e20091b1-c8b1-49e3-8a2b-5e9f217f98a4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.265861 Sep 30 09:51:54.369486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e27fca06-b272-4753-88b1-87da88c88651 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:54.370682 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e27fca06-b272-4753-88b1-87da88c88651 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:54.387279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 323/658] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:51:53 2022] POST /compute/v2.1/servers => generated 405 bytes in 1267 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:51:54.455102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e27fca06-b272-4753-88b1-87da88c88651 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:54.462603 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e27fca06-b272-4753-88b1-87da88c88651 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1095 microversion: 2.1 time: 0.097514 Sep 30 09:51:54.470702 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 336/659] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:54 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1095 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:55.489391 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-305e18f3-f754-40a8-b6f6-1096201e4d13 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:55.489993 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-305e18f3-f754-40a8-b6f6-1096201e4d13 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:55.492102 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=4,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:55.556438 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-305e18f3-f754-40a8-b6f6-1096201e4d13 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:55.564276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-305e18f3-f754-40a8-b6f6-1096201e4d13 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1095 microversion: 2.1 time: 0.079382 Sep 30 09:51:55.571724 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 324/660] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:55 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1095 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:56.165733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2139631805", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d038a068-f643-497f-8e6f-7d6e8fb5892b"}], "security_groups": [{"name": "tempest-securitygroup--1266574320"}], "key_name": "tempest-keypair-478929732", "user_data": "}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:56.329842 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] validate_networks() for [('d038a068-f643-497f-8e6f-7d6e8fb5892b', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:51:56.591585 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:56.592257 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:56.600094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:56.600731 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:56.601527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:56.602492 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:56.603605 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:56.604137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:56.708775 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:56.722073 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-def32086-a560-4854-8429-812eae75090c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1145 microversion: 2.1 time: 0.138380 Sep 30 09:51:56.723919 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 325/661] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:56 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1145 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:51:57.069840 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:51:57.070269 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:51:57.070792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:51:57.071202 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:51:57.071615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:51:57.082725 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:57.083507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:57.084118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:57.084855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:57.085560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:57.085994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:57.103909 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for project 86e6b0501cb94028be07bddd2182638b. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:51:57.108530 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Getting quotas for user 8c4ced79c3ff4842b30b472acb555cd5 and project 86e6b0501cb94028be07bddd2182638b. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:51:57.122433 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:51:57.125276 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:51:57.135923 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=3,nova_api:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:57.243409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-291ea9f3-d0db-47f8-b06a-f39fa53d8b44 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.082195 Sep 30 09:51:57.244223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 337/662] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:51:56 2022] POST /compute/v2.1/servers => generated 402 bytes in 1084 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:51:57.256522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9db2423a-0ce1-42ee-ba28-5a1930125c1f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:57.257211 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9db2423a-0ce1-42ee-ba28-5a1930125c1f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:57.326762 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9db2423a-0ce1-42ee-ba28-5a1930125c1f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:57.336165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9db2423a-0ce1-42ee-ba28-5a1930125c1f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1241 microversion: 2.1 time: 0.083444 Sep 30 09:51:57.337415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 326/663] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:57 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1241 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:57.360508 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:57.361113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:57.368187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:57.368679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:57.370496 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:57.371454 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:57.372079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:57.372652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:57.406033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lazy-loading 'flavor' on Instance uuid dcd3d415-addc-4607-844d-2ab1c2c9d16b {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:51:57.751469 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:57.773660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:57.784316 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-614620c5-17bc-40ac-b6f7-13454511d82c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "POST /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b/action" status: 202 len: 0 microversion: 2.1 time: 0.430629 Sep 30 09:51:57.785279 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 338/664] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:51:57 2022] POST /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b/action => generated 0 bytes in 434 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:51:57.798055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:57.802221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:57.804147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:57.805532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:57.805986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:57.806908 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:57.807276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:57.808087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:57.815177 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:57.815875 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:57.816320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:57.817029 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:57.817485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:57.817913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:57.846566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:57.935584 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:57.944593 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-55285195-4c3b-41eb-a6b7-945d32bda12d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1145 microversion: 2.1 time: 0.200006 Sep 30 09:51:57.945862 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 327/665] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:57 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1145 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:58.050584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:58.058657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-675b4e1b-adc8-420e-a815-d82f5f505bcd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.267138 Sep 30 09:51:58.059991 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 339/666] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:57 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:58.366503 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:58.367491 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:58.382482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:58.382482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:58.382482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:58.382482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:58.383396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:58.383396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:58.508092 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:58.517118 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-760b2232-ee36-426b-af38-45c1685dcd3e tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1245 microversion: 2.1 time: 0.155347 Sep 30 09:51:58.523148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 328/667] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:58 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1245 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:58.602542 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-2128042138", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "networks": [{"uuid": "70073525-5499-4f0a-9017-eb0ecf8cf458"}]}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:58.667783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] validate_networks() for [('70073525-5499-4f0a-9017-eb0ecf8cf458', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:51:58.958789 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7640c7dc-d425-4972-a1c1-eac58c4b6df3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'create', calling method: .version_select at 0x7f72c0b11ee0>, body: {"keypair": {"name": "tempest-keypair-1043075220", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBArYpL6U4WbmPPyYKmJ9hBwNxXkpxy0sCV/vPpebOy0Ivt5D2dppNnaI1Fa5ukcUTRsv2MYugZu9KY489cEz+1iS0AtiqViG3YC3FriWw1lI5d5DHaDhZeKB+S/62JddIw=="}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:51:58.981226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-7640c7dc-d425-4972-a1c1-eac58c4b6df3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Getting quotas for project 1142014cc9284cf1833e5bfbcdb41599. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:51:58.985310 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-7640c7dc-d425-4972-a1c1-eac58c4b6df3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Getting quotas for user 21d5e60badb846fb840a65f443728a9b and project 1142014cc9284cf1833e5bfbcdb41599. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:51:59.091446 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7640c7dc-d425-4972-a1c1-eac58c4b6df3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.293917 Sep 30 09:51:59.092663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 329/668] 10.222.0.39 () {62 vars in 1200 bytes} [Fri Sep 30 09:51:58 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 296 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 30 09:51:59.112537 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:59.113567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:59.127853 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:51:59.128270 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:51:59.128735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:51:59.129135 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:51:59.129575 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:51:59.129679 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:59.130132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:59.130564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:59.131300 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:59.131832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:59.132267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:59.139825 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:59.140771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:59.141681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:59.142868 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:59.143644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:59.144228 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:59.164167 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Getting quotas for project b3db64185ae94cefa4efa6f3e03c2f7e. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:51:59.168764 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Getting quotas for user 8f8db95b40ca4350b4f86ce9f1593194 and project b3db64185ae94cefa4efa6f3e03c2f7e. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:51:59.182685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:51:59.185162 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:51:59.201693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:59.209974 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ce7cbeed-e2a0-4b3d-89c0-232af0dc6242 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1193 microversion: 2.1 time: 0.116039 Sep 30 09:51:59.210550 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 330/669] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:59 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1193 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:59.217078 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:59.218083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:59.225267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:59.225916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:59.226627 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:59.227396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:59.227913 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:59.228399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:59.366992 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:59.375974 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-664a845c-cf51-4575-8783-a7d583cf4ce6 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.164332 Sep 30 09:51:59.376793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 331/670] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:59 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:59.402903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f3e04ba5-6eee-4f7d-9ebc-0f39a96803ea tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.804482 Sep 30 09:51:59.403937 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 340/671] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:51:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 806 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:51:59.419520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a6e3394f-b8ce-493e-b598-e8bdc77e4618 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:59.420861 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a6e3394f-b8ce-493e-b598-e8bdc77e4618 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:59.542966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:51:59.543704 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:51:59.551011 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:59.551576 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:59.552714 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:59.553793 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:51:59.554469 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:51:59.555364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:51:59.556376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a6e3394f-b8ce-493e-b598-e8bdc77e4618 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:59.564682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a6e3394f-b8ce-493e-b598-e8bdc77e4618 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1219 microversion: 2.1 time: 0.149487 Sep 30 09:51:59.565283 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 332/672] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:59 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1219 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:51:59.667698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:51:59.675825 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-17ead2df-41d7-41d2-8c5a-3b5d46588d4c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1237 microversion: 2.1 time: 0.137279 Sep 30 09:51:59.681379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 341/673] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:51:59 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1237 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:00.235331 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:00.236506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:00.244619 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:00.246074 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:00.246705 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:00.247567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:00.248175 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:00.248753 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:00.339764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:00.346156 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:00.348950 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9c9ddbfd-4dbc-45ac-b8dc-3207810bf142 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1211 microversion: 2.1 time: 0.117770 Sep 30 09:52:00.349636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 333/674] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:00 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1211 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:00.400530 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:00.401772 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:00.430697 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:00.431254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:00.431709 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:00.432535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:00.432995 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:00.433423 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:00.591289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:00.594111 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:00.609601 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:00.610443 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:00.611068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:00.611971 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:00.631905 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:00.632672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:00.685669 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:00.693497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-04adebfd-0ca8-46ac-b9d7-1c6ad0d7fca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.296726 Sep 30 09:52:00.702511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 342/675] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:00 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:00.720901 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:00.720901 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:00.722177 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:00.722705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:00.723149 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:00.723865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:00.724322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:00.738865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:00.771698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:00.783782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-044577c7-c8ab-4a31-8a33-9009ececebdd tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1223 microversion: 2.1 time: 0.198415 Sep 30 09:52:00.784734 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 334/676] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:00 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1223 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:00.816073 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:00.824604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ceec46c3-d4ec-4420-9199-84c5e2dc50e4 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1301 microversion: 2.1 time: 0.120902 Sep 30 09:52:00.831372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 343/677] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:00 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1301 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:01.390320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:01.391304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:01.410010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:01.410497 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:01.410976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:01.412379 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:01.412994 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:01.413570 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:01.732704 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:01.733464 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:01.941456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:01.941456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:01.941456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:01.941456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:01.942348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:01.942348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:01.943237 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:01.943237 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5e447e01-da44-431f-9e18-4a689fcf3263 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1278 microversion: 2.1 time: 0.422406 Sep 30 09:52:01.943237 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 335/678] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:01 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1278 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:01.943237 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:01.943931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:01.943931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:01.943931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:01.943931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:01.944491 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:01.944491 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:01.944491 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:01.944713 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:01.952148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5a35963c-377b-4db2-98b1-0e67e5b5efaa tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1271 microversion: 2.1 time: 0.144877 Sep 30 09:52:01.953214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 336/679] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:01 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1271 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:01.957861 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:01.958456 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:01.965621 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:01.966103 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:01.966533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:01.967881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:01.967881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:01.968244 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:02.157226 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:02.171305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c36c1579-397e-4a47-8b90-1d91f05d8cce tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.443336 Sep 30 09:52:02.186782 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 344/680] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:01 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:02.397131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:02.404374 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-abc77d2c-093f-4a3f-b827-126aa446eac6 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.450006 Sep 30 09:52:02.404959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 337/681] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:01 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:02.825373 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:02.826575 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:02.872570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:02.873099 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:02.873549 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:02.874572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:02.875066 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:02.875493 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:02.984823 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:02.985506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:02.993893 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:02.994393 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:02.995583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:02.996473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:02.996930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:02.997358 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:03.140915 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:03.148323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-bf4c7380-aa7f-41e0-8065-f559fd2fe989 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1289 microversion: 2.1 time: 0.171035 Sep 30 09:52:03.148896 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 338/682] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:02 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1289 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:03.154345 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-257761643", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6555801d-cf6b-484f-a48c-039d4395e1ca"}], "security_groups": [{"name": "tempest-securitygroup--32578007"}], "key_name": "tempest-keypair-1043075220", "user_data": "}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:03.249878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:03.257853 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d64cd19d-7396-4154-865b-8f9b37e70537 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.437142 Sep 30 09:52:03.258546 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 345/683] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:02 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:03.263196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:03.264064 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:03.276840 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:03.276840 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:03.276840 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:03.276840 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:03.277310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:03.277310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:03.394012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.network.neutron [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] validate_networks() for [('6555801d-cf6b-484f-a48c-039d4395e1ca', None, None, None, None, None)] {{(pid=127455) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:52:03.660475 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:03.668926 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-53d7101c-8bc9-442f-a768-4c0b6b91b801 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.409298 Sep 30 09:52:03.669511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 346/684] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:03 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:03.678546 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:03.680446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:03.687787 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:03.688312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:03.688774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:03.689472 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:03.689922 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:03.690343 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:03.813308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=2 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:04.058959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:04.069033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0139cec0-486b-4c87-b0bd-0452049595af tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.398091 Sep 30 09:52:04.078913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 347/685] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:03 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:04.086903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-7e536c20-dc75-412b-b17b-9e3395af03ea req-2b70fcb4-676a-42f0-9402-b6d9fac3d4fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bcd7505a-8f4d-4b0b-969e-776089b2e6d1", "tag": "c69dcca2-dd34-464e-98ba-2697ae2f9628"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:04.097579 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7e536c20-dc75-412b-b17b-9e3395af03ea req-2b70fcb4-676a-42f0-9402-b6d9fac3d4fe service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.098136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7e536c20-dc75-412b-b17b-9e3395af03ea req-2b70fcb4-676a-42f0-9402-b6d9fac3d4fe service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.098569 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7e536c20-dc75-412b-b17b-9e3395af03ea req-2b70fcb4-676a-42f0-9402-b6d9fac3d4fe service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.110265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-7e536c20-dc75-412b-b17b-9e3395af03ea req-2b70fcb4-676a-42f0-9402-b6d9fac3d4fe service nova] Creating event network-changed:c69dcca2-dd34-464e-98ba-2697ae2f9628 for instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 on hv-cinder-maste Sep 30 09:52:04.115388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-7e536c20-dc75-412b-b17b-9e3395af03ea req-2b70fcb4-676a-42f0-9402-b6d9fac3d4fe service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034533 Sep 30 09:52:04.115979 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 348/686] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:04.121855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-39a2d930-d4a1-44ae-b86c-fd6b7595784b req-5392f9cf-c08a-4c24-9cfb-6a0afaabead3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ff7e4e2-632b-402b-b30d-c0916cb3b498", "tag": "c95b82c7-207b-4532-98c9-cb356b1ea7fd"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:04.132426 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-39a2d930-d4a1-44ae-b86c-fd6b7595784b req-5392f9cf-c08a-4c24-9cfb-6a0afaabead3 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.132948 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-39a2d930-d4a1-44ae-b86c-fd6b7595784b req-5392f9cf-c08a-4c24-9cfb-6a0afaabead3 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.133372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-39a2d930-d4a1-44ae-b86c-fd6b7595784b req-5392f9cf-c08a-4c24-9cfb-6a0afaabead3 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.145946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-39a2d930-d4a1-44ae-b86c-fd6b7595784b req-5392f9cf-c08a-4c24-9cfb-6a0afaabead3 service nova] Creating event network-changed:c95b82c7-207b-4532-98c9-cb356b1ea7fd for instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 on hv-cinder-maste Sep 30 09:52:04.150676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-39a2d930-d4a1-44ae-b86c-fd6b7595784b req-5392f9cf-c08a-4c24-9cfb-6a0afaabead3 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033215 Sep 30 09:52:04.152264 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 349/687] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:04.172790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:04.174097 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:04.180174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.180904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.181569 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.182802 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.183520 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.184085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.302497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:04.310940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d6ad6a52-df63-40a1-9f64-8789ac9d524a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1277 microversion: 2.1 time: 0.142208 Sep 30 09:52:04.311905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 350/688] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:04 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1277 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:04.322583 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:04.324093 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:04.333157 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.333903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.334347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.335370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.336043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.336478 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.373972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Flavor limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:52:04.373972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Image limits 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:52:04.373972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Flavor pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:52:04.373972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Image pref 0:0:0 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:52:04.373972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.virt.hardware [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127455) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:52:04.382079 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.382556 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.383041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.383782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.384224 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.384635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.403972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Getting quotas for project 1142014cc9284cf1833e5bfbcdb41599. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:52:04.408545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Getting quotas for user 21d5e60badb846fb840a65f443728a9b and project 1142014cc9284cf1833e5bfbcdb41599. Resources: {'ram', 'instances', 'cores'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:52:04.419768 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Going to run 1 instances... {{(pid=127455) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:52:04.422156 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127455) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:52:04.581816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-20c2c381-00d0-41ca-acc1-49bbdc562b39 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 1.431701 Sep 30 09:52:04.583090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 339/689] 10.222.0.39 () {62 vars in 1188 bytes} [Fri Sep 30 09:52:03 2022] POST /compute/v2.1/servers => generated 400 bytes in 1433 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 30 09:52:04.595261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1ae76e6e-85ae-424f-a4d2-c6c9ed7bbc6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:04.596137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1ae76e6e-85ae-424f-a4d2-c6c9ed7bbc6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:04.809512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1ae76e6e-85ae-424f-a4d2-c6c9ed7bbc6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:04.819220 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1ae76e6e-85ae-424f-a4d2-c6c9ed7bbc6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:04.820847 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1ae76e6e-85ae-424f-a4d2-c6c9ed7bbc6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:04.822283 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1ae76e6e-85ae-424f-a4d2-c6c9ed7bbc6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1281 microversion: 2.20 time: 0.231176 Sep 30 09:52:04.831033 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 340/690] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:04 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1281 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:04.865300 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:04.866359 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:04.875350 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.877133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.877768 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.878936 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:04.879774 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:04.880558 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:04.976422 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:04.985077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-64083f9b-c9c3-44cb-8f56-3cede2708f19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.671895 Sep 30 09:52:04.986002 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 351/691] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:04 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:05.056177 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:05.064675 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2d14099a-f4bf-4907-a920-0c47289e8469 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.231720 Sep 30 09:52:05.065254 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 341/692] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:04 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:05.092528 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:05.093135 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:05.100544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:05.101257 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:05.101688 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:05.102389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:05.102861 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:05.103460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:05.335325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:05.335699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:05.342361 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:05.343053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:05.343656 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:05.344433 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:05.344983 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:05.345682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:05.359764 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:05.386731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-442c5395-ddd7-46f4-833a-270e7ca0665f tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.297472 Sep 30 09:52:05.387905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 352/693] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:05 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:05.581835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:05.598737 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d178de41-b4f2-4fb9-b922-9fbbee3d399d tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.268255 Sep 30 09:52:05.599420 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 342/694] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:05 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:05.849942 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:05.850579 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:05.856617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:05.857103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:05.857535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:05.858231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:05.858713 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:05.859138 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:05.926703 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:05.935409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:05.936845 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:05.937967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-25cb7490-d57f-45f4-a7cf-c17afb066dd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1285 microversion: 2.20 time: 0.092331 Sep 30 09:52:05.938839 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 353/695] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:05 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1285 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:06.009507 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:06.010215 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:06.016179 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.016718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.019392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.019392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.019392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.019392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.089098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:06.089894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:06.096038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.096640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.097039 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.097841 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.098358 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.098878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.254810 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:06.262773 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e076b8d2-647e-4e16-a74c-d4a478c03755 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.257207 Sep 30 09:52:06.263713 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 343/696] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:06 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:06.289616 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:06.299190 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5849c3da-4f3c-4833-9839-51e5e9c5d0a2 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.214941 Sep 30 09:52:06.302478 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 354/697] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:06 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:06.370572 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-eeab3718-66fc-4c66-861c-cf9abb62ba3f req-ce944fd2-3fc0-422c-bc51-a254fb39172c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d748a97-9d22-4676-b253-1ff841288220", "tag": "ab1411b2-93da-4dea-aed5-b2432b7055d1"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:06.380373 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-eeab3718-66fc-4c66-861c-cf9abb62ba3f req-ce944fd2-3fc0-422c-bc51-a254fb39172c service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.380848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-eeab3718-66fc-4c66-861c-cf9abb62ba3f req-ce944fd2-3fc0-422c-bc51-a254fb39172c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.381273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-eeab3718-66fc-4c66-861c-cf9abb62ba3f req-ce944fd2-3fc0-422c-bc51-a254fb39172c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.392592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-eeab3718-66fc-4c66-861c-cf9abb62ba3f req-ce944fd2-3fc0-422c-bc51-a254fb39172c service nova] Creating event network-changed:ab1411b2-93da-4dea-aed5-b2432b7055d1 for instance 0d748a97-9d22-4676-b253-1ff841288220 on hv-cinder-maste Sep 30 09:52:06.398061 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-eeab3718-66fc-4c66-861c-cf9abb62ba3f req-ce944fd2-3fc0-422c-bc51-a254fb39172c service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037551 Sep 30 09:52:06.399231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 344/698] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:52:06.410917 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:06.411788 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:06.423155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.423155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.423155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.423155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.423523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.423523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.622316 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:06.623087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:06.631370 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.632371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.632806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.634155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.635764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.640123 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.644678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:06.654940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a1dbe38a-b824-45b8-8024-4f450cc3d526 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.247393 Sep 30 09:52:06.655122 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 355/699] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:06 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:06.867170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:06.877545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-71eccc6c-b46c-4b84-9fce-1b15ef112506 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.259223 Sep 30 09:52:06.883128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 345/700] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:06 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:06.964554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:06.965336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:06.973479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.975073 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.975073 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:06.975299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:06.975590 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:06.976020 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.092969 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:07.105354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:07.106160 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:07.107388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-008a14ca-59e8-4ba9-9c84-618ac8566068 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1333 microversion: 2.20 time: 0.147681 Sep 30 09:52:07.107940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 356/701] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:06 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1333 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:07.290275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:07.290835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:07.296726 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:07.297218 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:07.297647 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.298319 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:07.298794 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:07.299206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.323846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:07.324450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:07.330243 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:07.330758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:07.331191 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.331886 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:07.332364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:07.332783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.536344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:07.544498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-298ca8e2-89be-473e-a1ac-267c68c4ce03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.258048 Sep 30 09:52:07.544987 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 346/702] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:07 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:07.585347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:07.593355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c7b02923-a4ae-4ec4-9aa9-0eb34f0b0760 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.273990 Sep 30 09:52:07.593918 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 357/703] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:07 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:07.677225 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:07.677982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:07.685320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:07.687877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:07.687877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.687877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:07.687877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:07.688712 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.820766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:07.828725 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6058e69a-10e0-421d-a826-cd47853825d0 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.155640 Sep 30 09:52:07.829277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 347/704] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:07 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:07.901738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:07.902341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:07.908303 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:07.908782 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:07.909215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.909896 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:07.910340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:07.910790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:07.914798 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=100 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:08.035758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:08.043808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b0decbeb-de62-497c-bf32-540a845f5138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.146412 Sep 30 09:52:08.044365 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 358/705] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:07 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:08.135411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:08.135944 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:08.141885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:08.142365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:08.142878 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:08.143597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:08.144052 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:08.144494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:08.205924 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:08.214775 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:08.216251 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:08.217339 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-dc1f38b4-4c2d-4672-b7e1-274519b0cf84 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1351 microversion: 2.20 time: 0.085914 Sep 30 09:52:08.217891 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 348/706] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:08 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1351 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:08.568490 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:08.569194 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:08.575667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:08.576154 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:08.576674 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:08.577277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:08.577734 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:08.578183 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:08.615887 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:08.617172 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:08.624772 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:08.625651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:08.626307 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:08.627397 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:08.628394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:08.628882 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:08.873676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:08.882725 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bda666ee-5568-4630-ada6-fda64c7a805a tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.318136 Sep 30 09:52:08.883150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:08.886529 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 359/707] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:08 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:08.897978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f955a148-a9bb-4fd8-adc1-36e452628ca3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.285599 Sep 30 09:52:08.899894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 349/708] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:08 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:08.909051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:08.909863 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:08.916287 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:08.917049 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:08.917678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:08.918647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:08.919266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:08.919855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:09.068471 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:09.070025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:09.077293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:09.078325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:09.082689 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:09.082689 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:09.082689 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:09.082689 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:09.126945 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:09.134642 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-702972fe-fc0b-48c0-936a-417b2bcd5688 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.232410 Sep 30 09:52:09.135294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 360/709] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:08 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:09.244382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:09.249197 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:09.258954 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=4 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:09.263040 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=18 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:09.265293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:09.265968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:09.267165 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:09.268079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:09.268700 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:09.269288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:09.304881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:09.320607 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2e947062-7af3-4462-b43e-75050509146a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.257800 Sep 30 09:52:09.321208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 350/710] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:09 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:09.501994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:09.514741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:09.516204 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:09.517396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b8f43eba-5a0f-4d6f-8271-37fe2c0ca819 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.278420 Sep 30 09:52:09.518880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 361/711] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:09 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:52:09.908267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:09.908882 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:09.915282 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:09.915770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:09.916202 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:09.916900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:09.917356 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:09.917779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:09.924972 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:09.926133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:09.932139 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:09.932660 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:09.933165 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:09.933781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:09.934236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:09.934698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.125196 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:10.133327 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-daa8eaae-bc2b-46c2-8777-86c3c3d71091 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.228868 Sep 30 09:52:10.133910 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 351/712] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:09 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:10.137283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:10.146912 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b3c1494c-f6d0-4c2f-9909-7ff9af4f26e1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.225659 Sep 30 09:52:10.148277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 362/713] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:09 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:10.157326 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:10.158008 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:10.166419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:10.166951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:10.167381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.168089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:10.168538 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:10.169137 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.306068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:10.309358 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:10.316206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-cd0af8a6-6660-45fb-bb36-529c0a8f5786 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.162661 Sep 30 09:52:10.316916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 352/714] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:10 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:10.345082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:10.345990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:10.352332 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:10.353061 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:10.353479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.354207 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:10.354877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:10.355334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.470889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-130a6db5-28d0-4a36-90ad-fc3f228ae26f req-ce3ed695-1b7e-4962-94ef-1a4d4aa234d3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dbe90c99-924b-4f03-85d5-4bfa0a6cd051", "tag": "869788d7-cd2f-4701-b6bd-a6f79a97b732"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:10.487325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-130a6db5-28d0-4a36-90ad-fc3f228ae26f req-ce3ed695-1b7e-4962-94ef-1a4d4aa234d3 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:10.488323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-130a6db5-28d0-4a36-90ad-fc3f228ae26f req-ce3ed695-1b7e-4962-94ef-1a4d4aa234d3 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:10.494618 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-130a6db5-28d0-4a36-90ad-fc3f228ae26f req-ce3ed695-1b7e-4962-94ef-1a4d4aa234d3 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.513315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:10.513844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-130a6db5-28d0-4a36-90ad-fc3f228ae26f req-ce3ed695-1b7e-4962-94ef-1a4d4aa234d3 service nova] Creating event network-changed:869788d7-cd2f-4701-b6bd-a6f79a97b732 for instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 on hv-cinder-maste Sep 30 09:52:10.521723 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-130a6db5-28d0-4a36-90ad-fc3f228ae26f req-ce3ed695-1b7e-4962-94ef-1a4d4aa234d3 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056522 Sep 30 09:52:10.524485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 353/715] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:10.531078 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e90eb14b-d2d9-4c27-a5e7-303a4da8eec9 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.187070 Sep 30 09:52:10.532315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 363/716] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:10 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:10.545845 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:10.546540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:10.554770 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:10.555510 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:10.555941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.556650 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:10.557266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:10.558077 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.708366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-eeb0f0e7-b063-4d19-b18c-19174635a72a req-237c6204-4648-4729-9184-03f9bf37c543 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ff7e4e2-632b-402b-b30d-c0916cb3b498", "name": "network-vif-plugged", "status": "completed", "tag": "c95b82c7-207b-4532-98c9-cb356b1ea7fd"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:10.718962 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-eeb0f0e7-b063-4d19-b18c-19174635a72a req-237c6204-4648-4729-9184-03f9bf37c543 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:10.719666 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-eeb0f0e7-b063-4d19-b18c-19174635a72a req-237c6204-4648-4729-9184-03f9bf37c543 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:10.720097 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-eeb0f0e7-b063-4d19-b18c-19174635a72a req-237c6204-4648-4729-9184-03f9bf37c543 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:10.727630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:10.742798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:10.744250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:10.746013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-eeb5ee5d-1d5f-45c9-9538-386ff3624d91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.204915 Sep 30 09:52:10.746561 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-eeb0f0e7-b063-4d19-b18c-19174635a72a req-237c6204-4648-4729-9184-03f9bf37c543 service nova] Creating event network-vif-plugged:c95b82c7-207b-4532-98c9-cb356b1ea7fd for instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 on hv-cinder-maste Sep 30 09:52:10.746734 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 354/717] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:10 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:10.766365 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-eeb0f0e7-b063-4d19-b18c-19174635a72a req-237c6204-4648-4729-9184-03f9bf37c543 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.139251 Sep 30 09:52:10.767007 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 364/718] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:52:11.156746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:11.157352 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:11.163786 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.164255 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.164688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.165374 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.165829 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.166242 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.172693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:11.173390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:11.180125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.180771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.186132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.186132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.186132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.186132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.365503 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:11.369975 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:11.373510 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-5a82f733-587a-4b8c-862a-83f3f1a0b85b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.220555 Sep 30 09:52:11.374085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 355/719] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:11 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:11.378307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-69978485-8b8b-42dd-994e-8ceaabbd7d63 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.210398 Sep 30 09:52:11.378992 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:11.379189 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 365/720] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:11 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:11.380024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:11.386133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.386634 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.387132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.387816 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.388445 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.388922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.554832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:11.555466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:11.562076 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.562685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.563223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.563925 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.564395 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.564806 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.820374 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:11.831647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:11.837323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-67ee696c-e513-4953-8a56-cb107d81ab52 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.462147 Sep 30 09:52:11.838150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 356/721] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:11 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:11.843379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dd88ae0f-a6e0-4871-a78d-40f3b9aa5898 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.292591 Sep 30 09:52:11.844089 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 366/722] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:11 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:11.844275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:11.844895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:11.846862 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:11.851812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.852301 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.852746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:11.853434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:11.853886 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:11.854395 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.015209 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:12.023332 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:12.024766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:12.025846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-01ac1815-0baa-4267-ad87-6e190ac70017 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.186600 Sep 30 09:52:12.026481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 357/723] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:11 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:12.395776 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:12.398457 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:12.401874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:12.402474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:12.403699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.404305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.404934 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.405640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.406105 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.406528 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.409231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.409711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.410220 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.410896 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.411340 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.411757 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.611335 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:12.615162 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:12.621376 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4fd03f3c-44e3-4444-837f-9354754adecd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.229356 Sep 30 09:52:12.625306 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-850ec7c0-c890-4c00-9777-5e0f79e211af tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.227644 Sep 30 09:52:12.627326 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 367/724] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:12 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:12.646651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 358/725] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:12 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:12.781274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-6775bb50-cda4-45a7-95e5-8300846ac2ae req-67a15c1a-6b83-4ea0-838e-877e555a0351 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d748a97-9d22-4676-b253-1ff841288220", "name": "network-vif-plugged", "status": "completed", "tag": "ab1411b2-93da-4dea-aed5-b2432b7055d1"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:12.790920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6775bb50-cda4-45a7-95e5-8300846ac2ae req-67a15c1a-6b83-4ea0-838e-877e555a0351 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.791398 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6775bb50-cda4-45a7-95e5-8300846ac2ae req-67a15c1a-6b83-4ea0-838e-877e555a0351 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.791827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6775bb50-cda4-45a7-95e5-8300846ac2ae req-67a15c1a-6b83-4ea0-838e-877e555a0351 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.802824 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-6775bb50-cda4-45a7-95e5-8300846ac2ae req-67a15c1a-6b83-4ea0-838e-877e555a0351 service nova] Creating event network-vif-plugged:ab1411b2-93da-4dea-aed5-b2432b7055d1 for instance 0d748a97-9d22-4676-b253-1ff841288220 on hv-cinder-maste Sep 30 09:52:12.807504 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-6775bb50-cda4-45a7-95e5-8300846ac2ae req-67a15c1a-6b83-4ea0-838e-877e555a0351 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030047 Sep 30 09:52:12.808091 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 368/726] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:12.860761 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:12.861401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:12.865514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:12.866208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:12.867817 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.868303 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.868737 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.869447 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.869895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.870300 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.872332 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.872803 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.873231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:12.874499 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:12.875302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:12.875748 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.069716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:13.074754 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:13.078375 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-da46d482-11c5-4397-842b-7c7f25607c1c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1367 microversion: 2.1 time: 0.222597 Sep 30 09:52:13.079174 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 359/727] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:12 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1367 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:13.083953 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:13.084551 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:13.091197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:13.091872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:13.092470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.093334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:13.093780 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:13.094193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.094555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e764802c-9346-4036-a4f0-25917b573138 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.232780 Sep 30 09:52:13.095361 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 369/728] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:12 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:13.226323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:13.234333 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:13.235803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:13.236873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d1de98fd-f9dc-447a-b225-4f8df8dba379 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.156650 Sep 30 09:52:13.237419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 360/729] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:13 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:13.649314 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:13.649930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:13.656093 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:13.656569 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:13.656953 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:13.657427 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.657664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:13.657788 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:13.658110 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:13.658526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.663894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:13.664365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:13.664788 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.665464 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:13.665909 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:13.666322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.852219 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:13.857551 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:13.860469 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1ac4c570-2359-43bb-8cea-950778da3404 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1570 microversion: 2.1 time: 0.215023 Sep 30 09:52:13.861247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 370/730] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:13 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1570 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:13.865398 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0eb4c7ae-5b5b-4ae1-8b11-a3cc1754ffdd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.212176 Sep 30 09:52:13.866032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 361/731] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:13 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:13.883406 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:13.884005 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:13.891026 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:13.891695 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:13.892338 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.893301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:13.893875 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:13.894391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:13.937573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127456) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 30 09:52:14.101435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:14.102036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:14.108702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.109181 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.109609 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.110290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.110894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.111316 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.227067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-25aa0488-b91a-42bc-a4e1-a191eed1aa86 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "POST /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b/action" status: 202 len: 0 microversion: 2.1 time: 0.347378 Sep 30 09:52:14.228023 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 371/732] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:52:13 2022] POST /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b/action => generated 0 bytes in 349 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:52:14.232989 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:14.234045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:14.239739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.240127 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.240551 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.241603 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.241839 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.242295 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.247559 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:14.257623 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0b32d901-c153-42ca-802c-9018c1f07793 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1557 microversion: 2.1 time: 0.160064 Sep 30 09:52:14.258375 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 362/733] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:14 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1557 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:14.263610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:14.264287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:14.290634 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.293668 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.293668 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.293668 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.293668 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.294084 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.395802 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=2 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:14.486007 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:14.494113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-73344fd1-2197-4a17-885a-ff3c360744a0 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.265233 Sep 30 09:52:14.513483 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 372/734] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:14 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:14.520941 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:14.537238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:14.543161 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:14.543900 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.544405 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.544848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.546291 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.546808 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.547239 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.555783 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-013602e8-9df9-462c-b691-47c763d4d1d7 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1578 microversion: 2.1 time: 0.296232 Sep 30 09:52:14.557011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 363/735] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:14 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1578 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:14.566308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:14.566308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:14.570267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.570951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.571468 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.572255 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.573494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.574177 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.734745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9cfdbd66-e5d1-4521-bec3-164fee9f2443 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.176354 Sep 30 09:52:14.738703 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 364/736] 10.222.0.39 () {58 vars in 1275 bytes} [Fri Sep 30 09:52:14 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-interface => generated 284 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:14.900332 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=24 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:14.904986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:14.905602 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:14.911457 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.911932 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.912358 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.913044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:14.913609 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:14.914029 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:14.928051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:14.938388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:14.940012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:14.941951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-76826005-ec8a-4523-8630-fa8f73120353 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.426978 Sep 30 09:52:14.942786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 373/737] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:14 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:15.164320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:15.173882 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b1b61e02-ebee-4613-a5e4-bc467cc542d1 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.288510 Sep 30 09:52:15.192603 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 365/738] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:14 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:15.516886 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:15.517693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:15.524181 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:15.524699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:15.525130 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:15.525818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:15.526924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:15.527287 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:15.581606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:15.582356 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:15.589053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:15.590797 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:15.590797 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:15.590797 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:15.591057 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:15.592780 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:15.824249 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:15.832179 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-265e34a7-a1aa-4887-afee-72dd11a09b61 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1319 microversion: 2.1 time: 0.319174 Sep 30 09:52:15.832754 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 374/739] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:15 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1319 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:15.860473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:15.868748 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-19f5465d-0b8e-4c61-8bb1-3ddd2d9f11a5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1518 microversion: 2.1 time: 0.291295 Sep 30 09:52:15.869320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 366/740] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:15 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1518 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:15.969864 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:15.971098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:15.977330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:15.977895 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:15.978400 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:15.979247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:15.979773 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:15.980259 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:15.985508 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:16.105871 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-0002d706-3f8e-4066-b7cd-8c584f100012 req-ae909e05-86fb-4f06-9144-0f2c9ab1ed8f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ff7e4e2-632b-402b-b30d-c0916cb3b498", "tag": "c95b82c7-207b-4532-98c9-cb356b1ea7fd"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:16.116149 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0002d706-3f8e-4066-b7cd-8c584f100012 req-ae909e05-86fb-4f06-9144-0f2c9ab1ed8f service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:16.116724 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0002d706-3f8e-4066-b7cd-8c584f100012 req-ae909e05-86fb-4f06-9144-0f2c9ab1ed8f service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:16.117328 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0002d706-3f8e-4066-b7cd-8c584f100012 req-ae909e05-86fb-4f06-9144-0f2c9ab1ed8f service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:16.130142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-0002d706-3f8e-4066-b7cd-8c584f100012 req-ae909e05-86fb-4f06-9144-0f2c9ab1ed8f service nova] Creating event network-changed:c95b82c7-207b-4532-98c9-cb356b1ea7fd for instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 on hv-cinder-maste Sep 30 09:52:16.136080 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-0002d706-3f8e-4066-b7cd-8c584f100012 req-ae909e05-86fb-4f06-9144-0f2c9ab1ed8f service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033476 Sep 30 09:52:16.136760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 367/741] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:52:16.157070 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:16.165112 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:16.166581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:16.167705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c3defeab-c27c-4f8c-8ae7-1d255f9dd899 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.202974 Sep 30 09:52:16.168266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 375/742] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:15 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:16.198772 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:16.201345 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:16.205755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:16.206236 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:16.206682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:16.207371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:16.207814 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:16.208229 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:16.348926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:16.357748 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-76d21583-8494-43ee-9d56-4c8211a3ae87 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.164085 Sep 30 09:52:16.359767 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 368/743] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:16 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:16.857920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:16.858702 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:16.862270 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:INSERT=6,nova_api:SELECT=93,nova_api:UPDATE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:16.865994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:16.866418 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:16.866898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:16.867647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:16.868102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:16.868604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:16.892793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:16.893463 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:16.900043 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:16.900730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:16.901693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:16.902399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:16.902962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:16.903597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:17.405264 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:17.414045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-60a252db-8d48-446a-a213-fca79f3df3b2 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1507 microversion: 2.1 time: 0.562241 Sep 30 09:52:17.414813 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 376/744] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:16 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1507 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:17.425844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-94c9d77b-f05d-47d8-a8ad-9fa8303d3896 req-51639b3d-f1e0-402f-9b5a-f0a5b2c20f6a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bcd7505a-8f4d-4b0b-969e-776089b2e6d1", "name": "network-vif-plugged", "status": "completed", "tag": "c69dcca2-dd34-464e-98ba-2697ae2f9628"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:17.440387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-94c9d77b-f05d-47d8-a8ad-9fa8303d3896 req-51639b3d-f1e0-402f-9b5a-f0a5b2c20f6a service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:17.441798 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-94c9d77b-f05d-47d8-a8ad-9fa8303d3896 req-51639b3d-f1e0-402f-9b5a-f0a5b2c20f6a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:17.442451 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-94c9d77b-f05d-47d8-a8ad-9fa8303d3896 req-51639b3d-f1e0-402f-9b5a-f0a5b2c20f6a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:17.442935 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:17.455999 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e35aa8e8-cc1b-4892-817b-2d5ad889a6ff tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1520 microversion: 2.1 time: 0.567088 Sep 30 09:52:17.467534 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-94c9d77b-f05d-47d8-a8ad-9fa8303d3896 req-51639b3d-f1e0-402f-9b5a-f0a5b2c20f6a service nova] Creating event network-vif-plugged:c69dcca2-dd34-464e-98ba-2697ae2f9628 for instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 on hv-cinder-maste Sep 30 09:52:17.471143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 369/745] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:16 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1520 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:17.474522 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-94c9d77b-f05d-47d8-a8ad-9fa8303d3896 req-51639b3d-f1e0-402f-9b5a-f0a5b2c20f6a service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058227 Sep 30 09:52:17.477430 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:17.477921 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 377/746] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:17.478064 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:17.484327 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:17.484613 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:17.485041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:17.486204 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:17.487208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:17.487782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:17.497357 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:17.498953 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:17.508527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:17.509419 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:17.510806 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:17.513573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:17.514317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:17.515552 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:17.721991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:17.737189 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:17.741723 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:17.743695 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:17.748426 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-36ef7371-d3d2-4d2b-a45e-3c67d67f6f83 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.276122 Sep 30 09:52:17.749365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 370/747] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:17 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:17.754867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fa49bd88-7301-4551-85b2-d6fe5a4b2a79", "device": "/dev/sdb"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:17.758812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:17.763648 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:17.764435 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:17.764867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:17.765638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:17.766077 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:17.766488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:17.769442 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-58491ac6-1241-490b-828c-1bb06da58056 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.289773 Sep 30 09:52:17.770518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=100 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:17.771246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 378/748] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:17 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:18.483865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:18.484463 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:18.491177 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:18.491689 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:18.492104 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:18.492799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:18.493252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:18.493678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:18.626415 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:18.634413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-137593b2-ab30-4e5e-8e83-2423ca6f42b5 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.154612 Sep 30 09:52:18.635004 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 379/749] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:18 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:18.775885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:18.777089 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:18.783541 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:18.783778 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:18.784335 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:18.785094 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:18.785643 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:18.789572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:18.928907 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:18.931779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8501ed23-e514-4c98-a34d-e8a345175c9a tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "POST /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.180761 Sep 30 09:52:18.932423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 371/750] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:52:17 2022] POST /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220/os-volume_attachments => generated 194 bytes in 1182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:18.937425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:18.939372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:18.939520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:18.939617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:18.940761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9465ae05-1a9d-49bb-8ca0-158487c9eeea tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.168709 Sep 30 09:52:18.941559 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 380/751] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:18 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:52:18.944615 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:18.945026 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:18.945450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:18.946139 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:18.946733 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:18.947016 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:19.083546 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:19.091856 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d1347427-71c9-44ec-9eaa-6eaa19c4eff0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1266 microversion: 2.1 time: 0.158305 Sep 30 09:52:19.092364 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 372/752] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:18 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:19.500844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-b66e8f45-8365-487b-a980-bd7c8d472076 req-45581bdf-25fe-422c-9b54-661b7f64bc5d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dcd3d415-addc-4607-844d-2ab1c2c9d16b", "name": "network-vif-unplugged", "status": "completed", "tag": "803fcd8d-66b3-48a0-bc79-84c2e79f336c"}, {"server_uuid": "dbe90c99-924b-4f03-85d5-4bfa0a6cd051", "name": "network-vif-plugged", "status": "completed", "tag": "869788d7-cd2f-4701-b6bd-a6f79a97b732"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:19.511772 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-b66e8f45-8365-487b-a980-bd7c8d472076 req-45581bdf-25fe-422c-9b54-661b7f64bc5d service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:19.512248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-b66e8f45-8365-487b-a980-bd7c8d472076 req-45581bdf-25fe-422c-9b54-661b7f64bc5d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:19.520453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-b66e8f45-8365-487b-a980-bd7c8d472076 req-45581bdf-25fe-422c-9b54-661b7f64bc5d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:19.524667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-b66e8f45-8365-487b-a980-bd7c8d472076 req-45581bdf-25fe-422c-9b54-661b7f64bc5d service nova] Creating event network-vif-unplugged:803fcd8d-66b3-48a0-bc79-84c2e79f336c for instance dcd3d415-addc-4607-844d-2ab1c2c9d16b on hv-cinder-maste Sep 30 09:52:19.525174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-b66e8f45-8365-487b-a980-bd7c8d472076 req-45581bdf-25fe-422c-9b54-661b7f64bc5d service nova] Creating event network-vif-plugged:869788d7-cd2f-4701-b6bd-a6f79a97b732 for instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 on hv-cinder-maste Sep 30 09:52:19.531303 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-b66e8f45-8365-487b-a980-bd7c8d472076 req-45581bdf-25fe-422c-9b54-661b7f64bc5d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.034480 Sep 30 09:52:19.531889 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 381/753] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:19 2022] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:19.657338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:19.657925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:19.664207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:19.664678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:19.665103 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:19.665779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:19.666222 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:19.667175 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:19.796468 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:19.804087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-92f4e357-010e-441e-9912-914b69f7d6b9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.150565 Sep 30 09:52:19.804654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 373/754] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:19 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:19.966390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:19.967356 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:19.973898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:19.976368 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:19.976368 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:19.976368 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:19.976368 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:19.977236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:20.113089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:20.113838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:20.120110 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:20.120592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:20.121018 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:20.121919 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:20.122722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:20.123294 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:20.240192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:20.248885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:20.250638 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:20.252114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a2af1774-b489-4abf-8de1-e558969d5d59 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.290509 Sep 30 09:52:20.258084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 382/755] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:19 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:20.315634 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:20.324580 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4f40a0a4-c3b2-4e6d-ae84-fff6d3e86b45 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1454 microversion: 2.1 time: 0.215210 Sep 30 09:52:20.325135 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 374/756] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:20 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1454 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:20.346975 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:20.347583 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:20.353450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:20.353924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:20.354397 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:20.355115 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:20.355564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:20.355981 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:20.484385 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:20.492596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-01f58e38-9bd1-4fff-9bea-4bc2f82c5b88 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1454 microversion: 2.1 time: 0.149476 Sep 30 09:52:20.493152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 383/757] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:20 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1454 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:20.833624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:20.834786 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:20.846863 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:20.847708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:20.848477 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:20.849775 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:20.850609 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:20.851207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:21.004163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:21.018390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-60d86730-7022-45ce-a511-518231513f4f tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.185913 Sep 30 09:52:21.018390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 375/758] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:20 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:21.278170 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:21.278760 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:21.285392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:21.285887 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:21.286321 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:21.287056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:21.287511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:21.287939 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:21.423550 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:21.432477 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:21.434823 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:21.436056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-77880868-332e-4026-91c3-4a5508f26e66 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1405 microversion: 2.20 time: 0.162336 Sep 30 09:52:21.437002 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 384/759] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:21 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1405 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:22.035529 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:22.036855 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:22.042918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:22.043399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:22.043838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:22.044524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:22.044974 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:22.045389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:22.185927 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:22.194566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-91fe0872-1339-4d09-8cfc-b57f774496be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.162814 Sep 30 09:52:22.204309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 376/760] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:22 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:22.460969 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:22.461571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:22.468146 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:22.468671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:22.469274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:22.469968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:22.485697 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:22.485697 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:22.547885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-e39c7cdc-10f4-4d0a-9257-b0e81b1fa307 req-e111951c-c4e1-4ecd-8cf8-cd725f3e64ac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bcd7505a-8f4d-4b0b-969e-776089b2e6d1", "tag": "c69dcca2-dd34-464e-98ba-2697ae2f9628"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:22.558234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-e39c7cdc-10f4-4d0a-9257-b0e81b1fa307 req-e111951c-c4e1-4ecd-8cf8-cd725f3e64ac service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:22.558730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-e39c7cdc-10f4-4d0a-9257-b0e81b1fa307 req-e111951c-c4e1-4ecd-8cf8-cd725f3e64ac service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:22.559340 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-e39c7cdc-10f4-4d0a-9257-b0e81b1fa307 req-e111951c-c4e1-4ecd-8cf8-cd725f3e64ac service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:22.571190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-e39c7cdc-10f4-4d0a-9257-b0e81b1fa307 req-e111951c-c4e1-4ecd-8cf8-cd725f3e64ac service nova] Creating event network-changed:c69dcca2-dd34-464e-98ba-2697ae2f9628 for instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 on hv-cinder-maste Sep 30 09:52:22.576420 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-e39c7cdc-10f4-4d0a-9257-b0e81b1fa307 req-e111951c-c4e1-4ecd-8cf8-cd725f3e64ac service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032196 Sep 30 09:52:22.577210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 377/761] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:52:22.628818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:22.637288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:22.638748 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:22.639898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-18cd1c93-4367-4485-a916-2a13d139b211 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1598 microversion: 2.20 time: 0.182877 Sep 30 09:52:22.640649 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 385/762] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:52:22 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1598 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:52:22.664828 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:22.665396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:22.672292 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:22.672700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:22.673122 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:22.674054 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:22.674503 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:22.674950 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:22.790611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2ebce4e1-9faa-456c-93c7-1e4415647234 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.129917 Sep 30 09:52:22.791224 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 378/763] 10.222.0.39 () {60 vars in 1316 bytes} [Fri Sep 30 09:52:22 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-interface => generated 285 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:52:23.228778 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:23.230420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:23.244410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:23.245387 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:23.246014 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:23.246956 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:23.247575 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:23.247994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:23.400384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:23.408822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e049edcc-9dcc-4798-98c5-91609f5b15bf tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.185608 Sep 30 09:52:23.409392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 386/764] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:23 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:24.432894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:24.433499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:24.439579 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:24.440055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:24.440482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:24.441203 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:24.441655 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:24.442078 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:24.579894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:24.587871 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-23ee8eeb-4e5b-4a70-b20c-7cd1fbcac0eb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.159483 Sep 30 09:52:24.589095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 379/765] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:24 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:24.591144 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-6daf1c9d-e662-46cd-92f8-4f43184f6c29 req-abaeb12a-f734-4fac-befa-7164124a362c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dbe90c99-924b-4f03-85d5-4bfa0a6cd051", "tag": "869788d7-cd2f-4701-b6bd-a6f79a97b732"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:24.601042 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6daf1c9d-e662-46cd-92f8-4f43184f6c29 req-abaeb12a-f734-4fac-befa-7164124a362c service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:24.601521 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6daf1c9d-e662-46cd-92f8-4f43184f6c29 req-abaeb12a-f734-4fac-befa-7164124a362c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:24.601945 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6daf1c9d-e662-46cd-92f8-4f43184f6c29 req-abaeb12a-f734-4fac-befa-7164124a362c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:24.614120 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-6daf1c9d-e662-46cd-92f8-4f43184f6c29 req-abaeb12a-f734-4fac-befa-7164124a362c service nova] Creating event network-changed:869788d7-cd2f-4701-b6bd-a6f79a97b732 for instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 on hv-cinder-maste Sep 30 09:52:24.618958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-6daf1c9d-e662-46cd-92f8-4f43184f6c29 req-abaeb12a-f734-4fac-befa-7164124a362c service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031885 Sep 30 09:52:24.619589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 387/766] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:52:25.611153 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:25.611747 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:25.618228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:25.618746 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:25.619177 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:25.620540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:25.620989 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:25.621434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:25.753860 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:25.761494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7e7a559a-959b-45e0-ae56-b263e9571450 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.154262 Sep 30 09:52:25.762056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 380/767] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:25 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:26.785312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:26.786024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:26.792148 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:26.792629 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:26.793064 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:26.793783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:26.794237 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:26.794692 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:26.922806 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:26.930519 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ec7def3e-423b-4008-87f3-514345444892 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.149932 Sep 30 09:52:26.931115 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 388/768] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:26 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:27.953759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:27.954366 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:27.960824 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:27.961299 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:27.961729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:27.962409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:27.962898 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:27.963910 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:28.089234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:28.097132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-096c2e82-5e52-4c9a-9558-e7d9c2f3f7f9 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.147351 Sep 30 09:52:28.097697 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 381/769] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:27 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:29.119708 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:29.120225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:29.126842 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:29.127317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:29.127747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:29.128425 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:29.128880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:29.129305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:29.263947 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:29.271548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f1eddbae-64be-4d8c-b4da-518278f57ec1 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.155843 Sep 30 09:52:29.272101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 389/770] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:29 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:30.295603 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:30.296113 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:30.302461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:30.302964 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:30.303389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:30.304061 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:30.304504 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:30.304917 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:30.431706 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:30.439382 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9d422c88-5c9c-41bc-82c2-0ada03af9082 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.148178 Sep 30 09:52:30.439942 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 382/771] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:30 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:31.461894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:31.462506 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:31.469054 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:31.469530 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:31.469955 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:31.470662 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:31.471113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:31.471532 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:31.607650 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:31.615333 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d4e83122-bdbb-433e-a600-ec496e9905ad tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.157310 Sep 30 09:52:31.615898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 390/772] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:31 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:32.639439 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:32.640040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:32.646529 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:32.647050 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:32.647479 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:32.648166 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:32.648614 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:32.649034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:32.778666 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:32.786624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7c617dd9-e23e-43e0-8d0d-3f1a28a3235a tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.151557 Sep 30 09:52:32.787237 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 383/773] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:32 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:33.809926 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:33.810546 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:33.817023 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:33.817506 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:33.817942 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:33.818651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:33.819114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:33.819536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:33.951152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:33.958968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c9d94943-0f0a-44bd-92ea-7b67b5aaed8c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.152923 Sep 30 09:52:33.959469 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 391/774] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:33 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:52:34.983035 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:34.983680 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:34.990161 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:34.990671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:34.991114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:34.991803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:34.992256 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:34.992684 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:35.118778 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:35.126632 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2e7ca896-abb6-4835-8e75-9c2c0ffaf7fc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.148057 Sep 30 09:52:35.127204 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 384/775] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:34 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:36.149301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:36.149905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:36.156242 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:36.156713 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:36.157138 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:36.157844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:36.158289 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:36.158739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:36.285375 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:36.293404 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bb5a4849-6962-4cf0-93a5-01685ec08fbc tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.147987 Sep 30 09:52:36.293966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 392/776] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:36 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:36.813553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-afc87d80-c9fb-4f3e-b19d-d19a3fa0decb req-714f9c6b-b0a4-45ef-a079-b4b7f55a84cc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dcd3d415-addc-4607-844d-2ab1c2c9d16b", "name": "network-vif-plugged", "status": "completed", "tag": "803fcd8d-66b3-48a0-bc79-84c2e79f336c"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:36.824066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-afc87d80-c9fb-4f3e-b19d-d19a3fa0decb req-714f9c6b-b0a4-45ef-a079-b4b7f55a84cc service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:36.824539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-afc87d80-c9fb-4f3e-b19d-d19a3fa0decb req-714f9c6b-b0a4-45ef-a079-b4b7f55a84cc service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:36.824961 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-afc87d80-c9fb-4f3e-b19d-d19a3fa0decb req-714f9c6b-b0a4-45ef-a079-b4b7f55a84cc service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:36.835832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-afc87d80-c9fb-4f3e-b19d-d19a3fa0decb req-714f9c6b-b0a4-45ef-a079-b4b7f55a84cc service nova] Creating event network-vif-plugged:803fcd8d-66b3-48a0-bc79-84c2e79f336c for instance dcd3d415-addc-4607-844d-2ab1c2c9d16b on hv-cinder-maste Sep 30 09:52:36.841286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-afc87d80-c9fb-4f3e-b19d-d19a3fa0decb req-714f9c6b-b0a4-45ef-a079-b4b7f55a84cc service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031481 Sep 30 09:52:36.841878 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 385/777] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:37.316046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:37.316691 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:37.323760 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:37.326512 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:37.326512 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:37.326512 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:37.326512 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:37.326908 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:37.510278 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:37.514084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=100 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:37.518702 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d6b8101a-9811-4bf9-83ce-85fb888f6ebb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.206502 Sep 30 09:52:37.519273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 393/778] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:37 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:38.542424 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:38.547293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:38.549227 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:38.549699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:38.550124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:38.550830 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:38.551277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:38.551693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:38.684017 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:38.691724 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-90b87538-b4c4-41b7-bcea-bdacb68609be tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1576 microversion: 2.1 time: 0.154091 Sep 30 09:52:38.692287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 386/779] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:38 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1576 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:39.717859 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:39.718466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:39.725364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:39.725846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:39.726278 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:39.726995 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:39.727450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:39.727873 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:39.856747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:39.864372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-cc3a6a91-b3c2-4c4a-8730-b12df30a1ae3 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1563 microversion: 2.1 time: 0.151022 Sep 30 09:52:39.864935 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 394/780] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:39 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1563 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:41.142196 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:41.142703 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:41.148929 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:41.149399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:41.149821 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:41.150499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:41.150967 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:41.151375 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:41.230254 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lazy-loading 'services' on Instance uuid 0d748a97-9d22-4676-b253-1ff841288220 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:52:41.233591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:41.418392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-182b59e6-a37e-4082-a61d-cd195fb61986 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "DELETE /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220/os-volume_attachments/fa49bd88-7301-4551-85b2-d6fe5a4b2a79" status: 202 len: 0 microversion: 2.1 time: 0.280391 Sep 30 09:52:41.419012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 387/781] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:52:41 2022] DELETE /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220/os-volume_attachments/fa49bd88-7301-4551-85b2-d6fe5a4b2a79 => generated 0 bytes in 282 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:52:43.599227 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:43.599720 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:43.605877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:43.606353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:43.606814 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:43.607497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:43.608486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:43.608907 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:43.636323 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:52:43.914676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a332e9c6-f80d-468a-8ece-a4be2d8e26ae tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "DELETE /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 204 len: 0 microversion: 2.1 time: 0.319049 Sep 30 09:52:43.915203 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 395/782] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:52:43 2022] DELETE /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:52:43.924928 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:43.925630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:43.927567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=62 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:43.932424 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:43.932899 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:43.933323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:43.933999 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:43.934439 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:43.934878 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:44.064776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:44.072353 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6988170c-6bf1-42e1-89a9-16331d368ef3 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1513 microversion: 2.1 time: 0.152773 Sep 30 09:52:44.072914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 388/783] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:43 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1513 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:45.095626 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:45.096322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:45.102871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:45.103346 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:45.103775 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:45.104457 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:45.104907 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:45.105330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:45.231475 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:45.239620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e0e69a4b-846d-48ed-a467-c6554260ab90 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1513 microversion: 2.1 time: 0.148244 Sep 30 09:52:45.240180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 396/784] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:45 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1513 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:46.263417 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:46.265202 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:46.271707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:46.272187 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:46.272616 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:46.273307 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:46.273757 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:46.274176 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:46.486015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:46.493393 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3de06896-eaac-4f49-8ba3-ba70930f65cb tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1513 microversion: 2.1 time: 0.233917 Sep 30 09:52:46.493956 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 389/785] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:46 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1513 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:46.784395 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-377d910d-37e4-42ea-9cad-7e295bc31a26 req-3cc3e338-8ca4-44b4-ba9c-4e3fdf692e6d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d748a97-9d22-4676-b253-1ff841288220", "name": "network-vif-deleted", "tag": "ab1411b2-93da-4dea-aed5-b2432b7055d1"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:52:46.793985 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-377d910d-37e4-42ea-9cad-7e295bc31a26 req-3cc3e338-8ca4-44b4-ba9c-4e3fdf692e6d service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:46.794462 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-377d910d-37e4-42ea-9cad-7e295bc31a26 req-3cc3e338-8ca4-44b4-ba9c-4e3fdf692e6d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:46.794924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-377d910d-37e4-42ea-9cad-7e295bc31a26 req-3cc3e338-8ca4-44b4-ba9c-4e3fdf692e6d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:46.806125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-377d910d-37e4-42ea-9cad-7e295bc31a26 req-3cc3e338-8ca4-44b4-ba9c-4e3fdf692e6d service nova] Creating event network-vif-deleted:ab1411b2-93da-4dea-aed5-b2432b7055d1 for instance 0d748a97-9d22-4676-b253-1ff841288220 on hv-cinder-maste Sep 30 09:52:46.810778 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-377d910d-37e4-42ea-9cad-7e295bc31a26 req-3cc3e338-8ca4-44b4-ba9c-4e3fdf692e6d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030075 Sep 30 09:52:46.811378 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 397/786] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:52:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:52:47.518154 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:47.518812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:47.525348 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:47.525823 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:47.526252 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:47.526971 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:47.527421 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:47.527839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:47.557529 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:47.558123 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:47.558553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:47.559020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lazy-loading 'fault' on Instance uuid 0d748a97-9d22-4676-b253-1ff841288220 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:52:47.598939 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8f8db95b40ca4350b4f86ce9f1593194', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3db64185ae94cefa4efa6f3e03c2f7e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:52:47.606405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-fbfed96c-2538-4dcf-a959-492b8c63910e tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 200 len: 1309 microversion: 2.1 time: 0.092900 Sep 30 09:52:47.607007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 390/787] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:47 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 1309 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:52:48.630468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:52:48.631205 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] [instance: 0d748a97-9d22-4676-b253-1ff841288220] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:52:48.633122 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:48.637938 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:48.638421 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:48.638888 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:48.639582 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:52:48.640038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:52:48.640953 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:52:48.653855 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] HTTP exception thrown: Instance 0d748a97-9d22-4676-b253-1ff841288220 could not be found. Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Request method failure captured: Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 HTTP/1.1 Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrvjxVxMeUAW3mNXYZq8IAl8AeycXWzOw699KjdLi7tX7ZycT1K2C1FqjzLqyayOYru21geWzNDOO49rLGiDgJwJ0tmeJQUIbqR0Fwx_tsx2csk4xGdb7WyvnG7HiwMWWCzWW-CuV3Qp3OIgIrmkFE01ONfi0ApYWQnu7ADC4yI8pho Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: b3db64185ae94cefa4efa6f3e03c2f7e Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1827020212 Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:52:48.655474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b3db64185ae94cefa4efa6f3e03c2f7e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b3db64185ae94cefa4efa6f3e03c2f7e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b3db64185ae94cefa4efa6f3e03c2f7e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b3db64185ae94cefa4efa6f3e03c2f7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-VolumesSnapshotTestJSON-1827020212 Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: b3db64185ae94cefa4efa6f3e03c2f7e Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1827020212 Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-VolumesSnapshotTestJSON-1827020212-project Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8f8db95b40ca4350b4f86ce9f1593194 Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-VolumesSnapshotTestJSON-1827020212-project Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 0d748a97-9d22-4676-b253-1ff841288220 could not be found. Sep 30 09:52:48.657740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '0d748a97-9d22-4676-b253-1ff841288220'} Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d748a97-9d22-4676-b253-1ff841288220 could not be found. Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:52:48.658794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d748a97-9d22-4676-b253-1ff841288220 could not be found. Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d748a97-9d22-4676-b253-1ff841288220 could not be found. Sep 30 09:52:48.660347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:52:48.662320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] Returning 404 to user: Instance 0d748a97-9d22-4676-b253-1ff841288220 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:52:48.662320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8287e336-e408-4235-9fe9-27be79440855 tempest-VolumesSnapshotTestJSON-1827020212 tempest-VolumesSnapshotTestJSON-1827020212-project] 10.222.0.39 "GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220" status: 404 len: 111 microversion: 2.1 time: 0.031968 Sep 30 09:52:48.662320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 398/788] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:52:48 2022] GET /compute/v2.1/servers/0d748a97-9d22-4676-b253-1ff841288220 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:53:01.555101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:01.555846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:01.557961 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=7 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:01.558692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=22,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:01.563436 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:01.563909 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:01.564376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:01.565054 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:01.565497 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:01.565912 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:01.589342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2a40697d-9138-4eb3-bea9-1d7461235f87 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/ips" status: 200 len: 150 microversion: 2.1 time: 0.038452 Sep 30 09:53:01.589889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 391/789] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 09:53:01 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/ips => generated 150 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:53:04.237394 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c7837308-8183-4208-b2f8-486a31376875"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:53:04.239749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:04.242678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=30,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:04.250160 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:04.250890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:04.264679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:04.264679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:04.264679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:04.264679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:04.921325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b4612362-e1c4-4a3c-8f7d-c3bbc753f408 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "POST /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.688262 Sep 30 09:53:04.921928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 399/790] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:53:04 2022] POST /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments => generated 194 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:53:13.611766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:13.612291 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:13.619533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:13.620019 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:13.620456 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:13.621145 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:13.621598 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:13.622045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:13.701305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lazy-loading 'services' on Instance uuid 4ff7e4e2-632b-402b-b30d-c0916cb3b498 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:53:13.893750 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f28d1de2-88da-4930-acc1-5acf6059b91c tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments/c7837308-8183-4208-b2f8-486a31376875" status: 202 len: 0 microversion: 2.1 time: 0.285601 Sep 30 09:53:13.894390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 392/791] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:53:13 2022] DELETE /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments/c7837308-8183-4208-b2f8-486a31376875 => generated 0 bytes in 288 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:53:13.903478 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:13.905885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:13.912514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:13.912997 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:13.913431 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:13.914123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:13.914615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:13.915039 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:13.943864 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ec6bfd70-6284-40e8-8524-1cb99dc41651 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044810 Sep 30 09:53:13.944415 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 400/792] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:53:13 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:53:14.957960 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:14.960465 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:14.966696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:14.967178 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:14.967669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:14.968476 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:14.968801 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:14.969406 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:15.020175 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e8f67a7e-c659-4373-9a12-b2752ad8c028 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067430 Sep 30 09:53:15.020751 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 393/793] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:53:14 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:53:16.033171 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:16.035888 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:16.043358 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:16.043849 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:16.044278 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:16.045016 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:16.045497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:16.045911 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:16.078084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-996c025b-1130-445a-9cf0-e6cf58c20fbb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049131 Sep 30 09:53:16.078686 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 401/794] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 09:53:16 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 09:53:16.142214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:16.142767 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:16.148957 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:16.149430 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:16.149851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:16.150526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:16.150998 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:16.151411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:16.177079 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:53:16.442690 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c995e4c9-9144-4505-980a-0382323f4744 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 204 len: 0 microversion: 2.1 time: 0.303988 Sep 30 09:53:16.443225 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 394/795] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:53:16 2022] DELETE /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:53:16.451290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:16.451898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:16.458160 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:16.458664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:16.459095 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:16.459770 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:16.460218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:16.460637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:16.589252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:16.597755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-68af2fc4-b560-4d8c-8e68-977bd18344e3 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1682 microversion: 2.1 time: 0.150270 Sep 30 09:53:16.598318 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 402/796] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:16 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1682 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:53:17.621682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:17.622377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:17.628946 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:17.629425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:17.629856 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:17.630543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:17.631013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:17.631433 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:17.758985 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:17.766580 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d63186fe-cc8c-4460-a55c-0f6323e29f86 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1682 microversion: 2.1 time: 0.149115 Sep 30 09:53:17.767164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 395/797] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:17 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1682 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:53:18.789570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:18.790288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:18.796846 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:18.797330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:18.801360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:18.801360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:18.801360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:18.801360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:19.000636 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:19.008668 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2f940505-7fa9-4084-a582-0bd79aaf3fd2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1682 microversion: 2.1 time: 0.222975 Sep 30 09:53:19.009282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 403/798] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:18 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1682 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:53:19.061424 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:19.061944 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:19.068755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:19.072324 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:19.072324 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:19.072324 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:19.072324 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:19.072692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:19.094470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6571e3f5-303f-4376-a71c-179d664dda54 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/ips" status: 200 len: 153 microversion: 2.20 time: 0.037099 Sep 30 09:53:19.095068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 396/799] 10.222.0.39 () {60 vars in 1289 bytes} [Fri Sep 30 09:53:19 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/ips => generated 153 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:53:19.232041 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-ce160e74-564b-471b-b372-0fd72a872948 req-b14eb8f2-d16a-463c-a257-13ba811fd159 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ff7e4e2-632b-402b-b30d-c0916cb3b498", "name": "network-vif-deleted", "tag": "c95b82c7-207b-4532-98c9-cb356b1ea7fd"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:53:19.242332 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ce160e74-564b-471b-b372-0fd72a872948 req-b14eb8f2-d16a-463c-a257-13ba811fd159 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:19.242857 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ce160e74-564b-471b-b372-0fd72a872948 req-b14eb8f2-d16a-463c-a257-13ba811fd159 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:19.243287 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ce160e74-564b-471b-b372-0fd72a872948 req-b14eb8f2-d16a-463c-a257-13ba811fd159 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:19.259927 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-ce160e74-564b-471b-b372-0fd72a872948 req-b14eb8f2-d16a-463c-a257-13ba811fd159 service nova] Creating event network-vif-deleted:c95b82c7-207b-4532-98c9-cb356b1ea7fd for instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 on hv-cinder-maste Sep 30 09:53:19.270571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-ce160e74-564b-471b-b372-0fd72a872948 req-b14eb8f2-d16a-463c-a257-13ba811fd159 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042201 Sep 30 09:53:19.271632 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 404/800] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:53:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:53:20.032070 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:20.032679 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:20.039516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:20.040011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:20.040458 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:20.041157 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:20.041916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:20.042202 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:20.073986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:20.074563 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:20.074940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:20.075452 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lazy-loading 'fault' on Instance uuid 4ff7e4e2-632b-402b-b30d-c0916cb3b498 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:53:20.118966 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8c4ced79c3ff4842b30b472acb555cd5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e6b0501cb94028be07bddd2182638b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:20.126731 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-89eb1ee7-92d4-47be-a246-cb95c5214299 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 200 len: 1331 microversion: 2.1 time: 0.098437 Sep 30 09:53:20.133109 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 397/801] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:20 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 1331 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:53:21.151481 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:21.152125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:21.158196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:21.159203 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:21.159634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:21.160322 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:21.160774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:21.161200 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:21.174662 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 HTTP/1.1 Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:53:21.176271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:21.177904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '4ff7e4e2-632b-402b-b30d-c0916cb3b498'} Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:21.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:21.181053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:21.182790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:21.182790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0ba4e288-b3bd-4baf-ac46-b93617260946 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 404 len: 111 microversion: 2.1 time: 0.031113 Sep 30 09:53:21.182790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 405/802] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:21 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:53:21.186760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ee961eb0-8b35-4279-81a7-70165b4d9890 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '.version_select at 0x7f72c0aee8b0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:21.272243 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ee961eb0-8b35-4279-81a7-70165b4d9890 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-478929732" status: 202 len: 0 microversion: 2.1 time: 0.089323 Sep 30 09:53:21.272863 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 398/803] 10.222.0.39 () {58 vars in 1218 bytes} [Fri Sep 30 09:53:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-478929732 => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:53:21.326479 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:53:21.327029 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:21.335274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:21.335274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:21.335274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:21.335274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:21.335631 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:21.335631 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:21.358115 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lazy-loading 'flavor' on Instance uuid dbe90c99-924b-4f03-85d5-4bfa0a6cd051 {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:53:21.842237 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-807f6506-1b7f-4f42-a522-9545b5d5c6d3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/action" status: 202 len: 0 microversion: 2.20 time: 0.519537 Sep 30 09:53:21.842863 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 406/804] 10.222.0.39 () {62 vars in 1319 bytes} [Fri Sep 30 09:53:21 2022] POST /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/action => generated 0 bytes in 521 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:53:21.851533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:21.852212 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:21.859131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:21.859597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:21.860032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:21.860722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:21.861169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:21.861589 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:22.062156 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:22.071267 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:22.072756 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:22.073902 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0e695b67-4833-4974-905c-ddf1c501297c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.226342 Sep 30 09:53:22.074454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 399/805] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:21 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:23.101392 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:23.102047 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:23.108451 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:23.108940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:23.109375 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:23.110069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:23.110527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:23.110979 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:23.238699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:23.246989 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:23.248438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:23.249587 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ba6b20d2-1cac-4c1e-a76a-fe9e9cab5537 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.153612 Sep 30 09:53:23.250145 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 407/806] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:23 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:23.301482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:23.301973 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:23.308282 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:23.308757 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:23.309214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:23.309885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:23.310327 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:23.310776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:23.323531 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: DELETE /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 HTTP/1.1 Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:53:23.325128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.326672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '4ff7e4e2-632b-402b-b30d-c0916cb3b498'} Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:23.327555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.329059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.331193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:23.331193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-aa5bc28c-dd23-4dad-bece-3b94f97a8086 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 404 len: 111 microversion: 2.1 time: 0.028746 Sep 30 09:53:23.331193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 400/807] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:53:23 2022] DELETE /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:53:23.336363 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:23.336980 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 4ff7e4e2-632b-402b-b30d-c0916cb3b498] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:23.342851 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:23.343336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:23.343772 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:23.344465 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:23.344919 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:23.345344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:23.359429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 HTTP/1.1 Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:53:23.360971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.362634 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '4ff7e4e2-632b-402b-b30d-c0916cb3b498'} Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:23.364241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. Sep 30 09:53:23.365743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:23.367085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 4ff7e4e2-632b-402b-b30d-c0916cb3b498 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:23.367085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-223b16d0-e098-4040-b376-484b29519dc2 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498" status: 404 len: 111 microversion: 2.1 time: 0.029801 Sep 30 09:53:23.367085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 408/808] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:23 2022] GET /compute/v2.1/servers/4ff7e4e2-632b-402b-b30d-c0916cb3b498 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:53:24.275846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:24.276454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:24.282611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.283091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.283545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.284229 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.284673 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.285105 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.420823 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:24.429187 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:24.430625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:24.431962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-89ee7b1a-3e85-477d-9a9c-2d3b1bd70698 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.160060 Sep 30 09:53:24.434319 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 401/809] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:24 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:24.695265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:24.695769 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:24.702090 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.702572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.703072 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.703761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.704215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.704635 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.717713 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 HTTP/1.1 Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:53:24.719261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.720869 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '27d777c7-ef67-43f8-a47c-eafb863feb81'} Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:24.721693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.723044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.724302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:24.724302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-04bffc22-93c0-4be2-9d6e-823c36a58853 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 404 len: 111 microversion: 2.1 time: 0.029306 Sep 30 09:53:24.724302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 409/810] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:53:24 2022] DELETE /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:53:24.728833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:24.729446 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 27d777c7-ef67-43f8-a47c-eafb863feb81] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:24.735451 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.735934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.737048 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.737742 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.738193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.738645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.751570 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 HTTP/1.1 Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:53:24.753947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.755629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '27d777c7-ef67-43f8-a47c-eafb863feb81'} Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:24.756553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. Sep 30 09:53:24.758045 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.759500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 27d777c7-ef67-43f8-a47c-eafb863feb81 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:24.759500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ca885cb2-dd06-47f0-b497-e057e25f9532 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81" status: 404 len: 111 microversion: 2.1 time: 0.030360 Sep 30 09:53:24.759500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 402/811] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:24 2022] GET /compute/v2.1/servers/27d777c7-ef67-43f8-a47c-eafb863feb81 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 30 09:53:24.763556 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:24.764059 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:24.769890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.770373 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.770843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.771535 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.771990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.772417 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.785749 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed HTTP/1.1 Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:53:24.787301 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.788903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '59975d37-8db0-4019-9303-88b12ef029ed'} Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:24.789818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.791420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.792927 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:24.792927 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-6fb1501c-f6c8-46ca-a74b-b37798a021bd tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 404 len: 111 microversion: 2.1 time: 0.028887 Sep 30 09:53:24.792927 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 410/812] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:53:24 2022] DELETE /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:53:24.797034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:24.797662 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 59975d37-8db0-4019-9303-88b12ef029ed] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:24.803990 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.804461 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.804883 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.805564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:24.806004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:24.806418 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:24.819843 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed HTTP/1.1 Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:53:24.821311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '59975d37-8db0-4019-9303-88b12ef029ed'} Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:24.824013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. Sep 30 09:53:24.825511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:24.826991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 59975d37-8db0-4019-9303-88b12ef029ed could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:24.826991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7b0e9119-07d9-437d-a0da-54d1538b687b tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed" status: 404 len: 111 microversion: 2.1 time: 0.029405 Sep 30 09:53:24.826991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 403/813] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:24 2022] GET /compute/v2.1/servers/59975d37-8db0-4019-9303-88b12ef029ed => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 30 09:53:25.457988 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:25.458611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:25.464416 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:25.464891 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:25.465316 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:25.465992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:25.466445 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:25.466894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:25.597638 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:25.605664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:25.607110 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:25.608247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ae091ae9-e4b2-4c70-8180-fbf81f5d6ace tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.154709 Sep 30 09:53:25.608798 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 411/814] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:25 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:26.054788 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:26.055277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:26.061804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:26.062344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:26.062832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:26.063529 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:26.063982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:26.064409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:26.078308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: DELETE /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 HTTP/1.1 Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:53:26.079892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.081514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '0bc1e701-2247-4837-81c0-289806279c31'} Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:26.082524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.084144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.085673 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:26.085673 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0201097c-3707-4c81-90ee-95fc374fa473 tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "DELETE /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 404 len: 111 microversion: 2.1 time: 0.030419 Sep 30 09:53:26.085673 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 404/815] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:53:26 2022] DELETE /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:53:26.089424 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:26.090059 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] [instance: 0bc1e701-2247-4837-81c0-289806279c31] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:26.096464 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:26.096940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:26.097367 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:26.098044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:26.098491 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:26.098933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:26.113537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] HTTP exception thrown: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Request method failure captured: Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 HTTP/1.1 Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrrZasGBhA2_hUE8pyppkZvfWutDxFcorUz5lQRQoiYxP5oqUo6sOBFWtdY2RHo4R20bp_rasyfr3VYzExPHcTz9t7g51uynfHNK0JkOuY4Xd8JbpfSrrroLBnSdXSSTIcjzzZLSwWLXRxV524scnShKW-lP-6PY9hR0GCoOrRnxSgg Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:53:26.115082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/86e6b0501cb94028be07bddd2182638b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_86e6b0501cb94028be07bddd2182638b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 86e6b0501cb94028be07bddd2182638b Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-2023202984 Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 8c4ced79c3ff4842b30b472acb555cd5 Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeNegativeTest-2023202984-project Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.116657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '0bc1e701-2247-4837-81c0-289806279c31'} Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:53:26.117644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. Sep 30 09:53:26.119150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:53:26.120488 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] Returning 404 to user: Instance 0bc1e701-2247-4837-81c0-289806279c31 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:53:26.120488 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-585fb98f-efcb-4295-ab7c-2b4162542ebb tempest-AttachVolumeNegativeTest-2023202984 tempest-AttachVolumeNegativeTest-2023202984-project] 10.222.0.39 "GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31" status: 404 len: 111 microversion: 2.1 time: 0.030774 Sep 30 09:53:26.120488 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 412/816] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:53:26 2022] GET /compute/v2.1/servers/0bc1e701-2247-4837-81c0-289806279c31 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:53:26.633707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:26.634228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:26.640509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:26.640995 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:26.641425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:26.642116 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:26.642564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:26.643044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:26.864145 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:26.872208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:26.873634 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:26.874818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3e43e52d-3d38-4063-9a41-6b15a96aae8f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.245120 Sep 30 09:53:26.875374 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 405/817] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:26 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:27.899239 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:27.899849 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:27.906151 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:27.906663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:27.907104 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:27.907792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:27.908736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:27.909159 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:28.083258 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:28.092603 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:28.094308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:28.095619 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7620e1c9-fccf-486f-905e-9d5eabc5d822 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.200311 Sep 30 09:53:28.096755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 413/818] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:27 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:29.121718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:29.122719 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:29.130234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:29.130840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:29.131463 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:29.132608 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:29.133155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:29.133766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:29.271567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:29.280583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:29.282718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:29.283963 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1df7ea7a-3696-45b9-9e09-7ff1c76a88c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.166546 Sep 30 09:53:29.286684 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 406/819] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:29 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:30.310079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:30.310719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:30.317577 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:30.318050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:30.318482 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:30.324354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:30.324354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:30.324354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:30.447360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:30.456740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:30.458321 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:30.459534 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b42321c5-281f-4de4-a658-0c0af3233439 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.153491 Sep 30 09:53:30.460088 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 414/820] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:30 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:31.484164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:31.484798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:31.491722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:31.492201 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:31.492628 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:31.493477 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:31.493933 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:31.494349 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:31.625819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:31.634044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:31.635525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:31.636672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-76990654-f948-429e-8e9a-fc409afd8f42 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.156590 Sep 30 09:53:31.637226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 407/821] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:31 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:32.663061 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:32.664864 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:32.670102 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:32.670613 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:32.671050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:32.671743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:32.672199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:32.672626 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:32.874651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:32.883113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:32.884554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:32.885703 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-adbbf023-9444-420e-998a-fd2c00db991e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1744 microversion: 2.20 time: 0.226569 Sep 30 09:53:32.886256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 415/822] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:32 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1744 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:33.912107 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:33.912829 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:33.919248 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:33.919736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:33.920165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:33.920859 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:33.921309 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:33.921726 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:34.079191 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:34.087901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:34.089342 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:34.090496 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-63c78675-b9fa-4bd4-9d5f-f20f9132970d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1744 microversion: 2.20 time: 0.182842 Sep 30 09:53:34.096174 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 408/823] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:33 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1744 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:35.116887 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:35.118205 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:35.122095 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=36 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:35.126039 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:35.126838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:35.127752 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:35.128821 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:35.129611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:35.130140 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:35.154817 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:35.303724 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:35.316085 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:35.319390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:35.319390 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1793709f-c02c-4a32-a65e-dd1519083e3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.206187 Sep 30 09:53:35.319697 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 416/824] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:35 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:36.348750 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:36.349353 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:36.357512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:36.358274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:36.358742 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:36.359606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:36.360299 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:36.361017 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:36.368611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:36.557136 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:36.567011 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:36.571443 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:36.571443 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-fdc46f5f-a4a7-4efd-bb9b-31db97416b00 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.228745 Sep 30 09:53:36.571443 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 409/825] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:36 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:37.260950 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-c4fdd36b-9509-4700-b7b5-206ed55534f9 req-7bd2e7d4-5021-421a-9f7e-6c7be4f0703c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dbe90c99-924b-4f03-85d5-4bfa0a6cd051", "name": "network-vif-plugged", "status": "completed", "tag": "869788d7-cd2f-4701-b6bd-a6f79a97b732"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:53:37.287936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-c4fdd36b-9509-4700-b7b5-206ed55534f9 req-7bd2e7d4-5021-421a-9f7e-6c7be4f0703c service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:37.289895 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-c4fdd36b-9509-4700-b7b5-206ed55534f9 req-7bd2e7d4-5021-421a-9f7e-6c7be4f0703c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:37.290444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-c4fdd36b-9509-4700-b7b5-206ed55534f9 req-7bd2e7d4-5021-421a-9f7e-6c7be4f0703c service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:37.306862 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-c4fdd36b-9509-4700-b7b5-206ed55534f9 req-7bd2e7d4-5021-421a-9f7e-6c7be4f0703c service nova] Creating event network-vif-plugged:869788d7-cd2f-4701-b6bd-a6f79a97b732 for instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 on hv-cinder-maste Sep 30 09:53:37.314359 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-c4fdd36b-9509-4700-b7b5-206ed55534f9 req-7bd2e7d4-5021-421a-9f7e-6c7be4f0703c service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057542 Sep 30 09:53:37.315965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 417/826] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:53:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:53:37.599681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:37.601707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:37.608222 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:37.608773 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:37.609283 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:37.610095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:37.617088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:37.617088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:37.800144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:37.810290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:37.811855 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:37.813080 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-290889b5-c762-422a-92e6-ad6b49065cfe tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.219616 Sep 30 09:53:37.813627 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 410/827] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:37 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:38.842435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:38.843225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:38.850336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:38.850881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:38.851620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:38.852035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:38.852508 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:38.852931 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:38.995979 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:39.005676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:39.007182 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:39.008555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b5c08797-b67d-43a6-8feb-ce896af4dd6f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.170715 Sep 30 09:53:39.008954 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 418/828] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:38 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:40.034788 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:40.036081 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:40.042491 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:40.043101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:40.043617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:40.044389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:40.044920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:40.045422 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:40.197958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:40.212209 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:40.219694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:40.221998 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2bb15848-8f46-42e5-9d1a-4454addd14d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.191091 Sep 30 09:53:40.223066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 411/829] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:40 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:41.250256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:41.251024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:41.258196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:41.258826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:41.259504 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:41.260415 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:41.261074 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:41.261701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:41.404182 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:41.413615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:41.415446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:41.416763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e1c18d33-2cf6-4d9f-a6d3-afa88e0fc104 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.170471 Sep 30 09:53:41.417287 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 419/830] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:41 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:42.442654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:42.443202 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:42.449834 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:42.450315 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:42.450791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:42.451484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:42.451941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:42.452366 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:42.588203 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:42.596757 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:42.614042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:42.614042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2bb89592-17a1-4a87-959f-0672416df92c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.161337 Sep 30 09:53:42.614042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 412/831] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:42 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:43.629624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:43.630367 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:43.639456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:43.640203 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:43.640820 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:43.641627 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:43.642084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:43.642505 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:43.787173 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:43.797589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:43.799739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:43.801330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-14ba5836-18e9-4c96-97fb-faaaf6784785 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.178656 Sep 30 09:53:43.801895 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 420/832] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:43 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:44.826978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:44.827500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:44.829781 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=53,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:44.837694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:44.838198 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:44.838657 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:44.839378 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:44.839828 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:44.840457 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:44.986920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:44.996039 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:44.997737 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:44.999296 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-70929757-f1f3-4eba-bd55-80c23697e606 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.176492 Sep 30 09:53:45.000085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 413/833] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:44 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:53:46.029355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:46.030197 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:46.036573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:46.037059 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:46.037482 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:46.038176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:46.038649 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:46.039261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:46.192081 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:46.208690 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:46.210517 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:46.215010 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-629f62d4-6362-422d-9939-4a83d85e1c16 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.186656 Sep 30 09:53:46.215795 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 421/834] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:46 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:47.240321 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:47.240967 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:47.247597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:47.248083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:47.248517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:47.249208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:47.249664 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:47.250088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:47.389578 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:47.394479 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:47.395978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:47.397120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e8bea230-64f5-4fa2-92fd-35244d9decc7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1739 microversion: 2.20 time: 0.160817 Sep 30 09:53:47.397918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 414/835] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:47 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1739 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:48.422354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:48.422999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:48.431588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:48.431588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:48.431588 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:48.431910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:48.432364 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:48.432788 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:48.568611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:48.576803 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:48.578252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:48.579436 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-608fddb8-7686-44d9-886e-f001e8cec913 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1721 microversion: 2.20 time: 0.160936 Sep 30 09:53:48.579989 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 422/836] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:48 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1721 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:49.606138 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:49.607661 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:49.614255 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:49.614777 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:49.615207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:49.616102 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:49.616558 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:49.617004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:49.745858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:49.754788 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:49.756208 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:49.757350 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ac310d9b-9c7f-4bc2-bf26-a861931a2769 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1721 microversion: 2.20 time: 0.155103 Sep 30 09:53:49.757907 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 415/837] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:49 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:50.782624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:50.784066 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:50.790485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:50.794685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:50.794685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:50.794685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:50.794685 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:50.795296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:50.930182 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:50.938481 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:50.939988 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:50.941411 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-cc2fe914-4951-4ce1-9722-9bd474657472 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1724 microversion: 2.20 time: 0.162727 Sep 30 09:53:50.941971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 423/838] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:50 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1724 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:50.970139 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "952937cf-c588-47d4-9ede-7fd97eda1772"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:53:50.973069 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:50.979509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:50.979993 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:50.980425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:50.981273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:50.981721 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:50.982139 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:51.326552 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-095cd9c3-f037-4903-afe1-b429a2183b58 req-4b64ef3f-cc12-49d2-969b-6b13bb279b89 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dbe90c99-924b-4f03-85d5-4bfa0a6cd051", "name": "network-vif-unplugged", "status": "completed", "tag": "869788d7-cd2f-4701-b6bd-a6f79a97b732"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:53:51.337372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-095cd9c3-f037-4903-afe1-b429a2183b58 req-4b64ef3f-cc12-49d2-969b-6b13bb279b89 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:51.337833 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-095cd9c3-f037-4903-afe1-b429a2183b58 req-4b64ef3f-cc12-49d2-969b-6b13bb279b89 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:51.338264 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-095cd9c3-f037-4903-afe1-b429a2183b58 req-4b64ef3f-cc12-49d2-969b-6b13bb279b89 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:51.349348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.compute.server_external_events [req-095cd9c3-f037-4903-afe1-b429a2183b58 req-4b64ef3f-cc12-49d2-969b-6b13bb279b89 service nova] Unable to find a host for instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051. Dropping event network-vif-unplugged {{(pid=127456) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 30 09:53:51.350228 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-095cd9c3-f037-4903-afe1-b429a2183b58 req-4b64ef3f-cc12-49d2-969b-6b13bb279b89 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.027807 Sep 30 09:53:51.350853 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 424/839] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:53:51 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 29 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Sep 30 09:53:52.034548 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-807bfaa4-ff52-42b9-abb0-dbc73834ecc2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.068400 Sep 30 09:53:52.035199 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 416/840] 10.222.0.39 () {62 vars in 1364 bytes} [Fri Sep 30 09:53:50 2022] POST /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments => generated 188 bytes in 1070 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:53:52.159612 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:53:52.160263 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:52.166701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:52.167180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:52.167607 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:52.168289 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:52.168740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:52.169157 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:52.198625 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127456) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 30 09:53:52.410258 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5b64668f-8e62-4b2f-a51e-b7a0593768b5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/action" status: 202 len: 0 microversion: 2.20 time: 0.254943 Sep 30 09:53:52.410820 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 425/841] 10.222.0.39 () {62 vars in 1319 bytes} [Fri Sep 30 09:53:52 2022] POST /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/action => generated 0 bytes in 256 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:53:52.419172 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:52.419857 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:52.426584 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:52.427106 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:52.427524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:52.428210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:52.428710 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:52.429127 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:52.557831 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:52.566700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:52.568128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:52.569271 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-40ad765a-bd1c-43f5-bb8c-ddec3651b0d4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1750 microversion: 2.20 time: 0.154115 Sep 30 09:53:52.569913 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 417/842] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:52 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1750 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:53.596605 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:53.597218 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:53.603433 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:53.603917 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:53.604384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:53.605074 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:53.605556 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:53.606036 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:53.754281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:53.763609 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:53.765124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:53.766272 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ba6bcc99-88ae-4f61-a78c-5e3720d210d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1750 microversion: 2.20 time: 0.174665 Sep 30 09:53:53.766859 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 426/843] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:53 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1750 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:54.792419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:54.792954 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:54.799767 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:54.800332 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:54.800760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:54.801434 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:54.801877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:54.802286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:54.937672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:54.946839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:54.948418 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:54.949591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-eaadf0a5-6d30-4186-9fb6-74f1adcbc84d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1752 microversion: 2.20 time: 0.161144 Sep 30 09:53:54.950335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 418/844] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:54 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1752 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:55.979228 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:55.979843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:55.986172 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:55.986696 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:55.987138 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:55.987830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:55.988288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:55.988736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:56.121542 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:56.132216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:56.133675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:56.134862 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1cc4cfb3-7ae2-443f-bd86-68d1f965ad40 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.159544 Sep 30 09:53:56.141334 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 427/845] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:55 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:57.161310 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:57.161906 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:57.168411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:57.168885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:57.169543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:57.170236 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:57.170712 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:57.171128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:57.307665 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:57.318210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:57.320147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:57.321936 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d0efb352-42f9-4f1b-9af0-60609a061696 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.164912 Sep 30 09:53:57.322504 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 419/846] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:57 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:58.347747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:58.348349 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:58.358545 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:58.359212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:58.359659 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:58.360448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:58.361000 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:58.361525 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:58.515817 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:58.525990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:58.527533 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:58.528883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f26c7e8d-fd61-44fa-9f2d-047020272d82 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.185665 Sep 30 09:53:58.530185 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 428/847] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:58 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:53:59.562337 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:53:59.563123 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:53:59.569560 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:59.570047 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:59.570480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:59.571205 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:53:59.571659 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:53:59.574177 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:53:59.804959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:59.813893 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:59.815479 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:53:59.816844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ad1a819a-b83f-4e30-adfe-8d024c400ee9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.258872 Sep 30 09:53:59.817718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 420/848] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:53:59 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:00.843788 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:00.844468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:00.850719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:00.851216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:00.851731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:00.852439 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:00.852898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:00.853324 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:01.015025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:01.025123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:01.027045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:01.029714 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c4ce2fca-21a0-49d9-a6e4-578c4b9bc177 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.190020 Sep 30 09:54:01.031393 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 429/849] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:00 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:02.058385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:02.059165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:02.069937 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:02.070429 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:02.076001 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:02.079249 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:02.079735 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:02.080278 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:02.394195 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:02.406652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:02.412636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:02.414086 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0da35ae3-a871-4ced-a7c8-14499a011de3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.360570 Sep 30 09:54:02.418881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 421/850] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:02 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:03.453981 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:03.457002 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:03.466001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:03.466910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:03.467573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:03.468739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:03.469408 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:03.470069 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:03.787788 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:03.796743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:03.798266 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:03.808235 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ce116cf5-ab19-48a5-8310-994a927eec57 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.355735 Sep 30 09:54:03.822633 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 430/851] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:03 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:04.101577 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:04.101577 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:04.107888 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-9eb54933-d790-4538-b996-872e2a1b69b3 req-32b27bbf-f863-4145-867b-e4880e7e4cc7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dcd3d415-addc-4607-844d-2ab1c2c9d16b", "tag": "803fcd8d-66b3-48a0-bc79-84c2e79f336c"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:54:04.109740 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:04.115041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:04.115611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:04.116424 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:04.116967 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:04.117540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:04.122029 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-9eb54933-d790-4538-b996-872e2a1b69b3 req-32b27bbf-f863-4145-867b-e4880e7e4cc7 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:04.122836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-9eb54933-d790-4538-b996-872e2a1b69b3 req-32b27bbf-f863-4145-867b-e4880e7e4cc7 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:04.123551 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-9eb54933-d790-4538-b996-872e2a1b69b3 req-32b27bbf-f863-4145-867b-e4880e7e4cc7 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:04.137112 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-9eb54933-d790-4538-b996-872e2a1b69b3 req-32b27bbf-f863-4145-867b-e4880e7e4cc7 service nova] Creating event network-changed:803fcd8d-66b3-48a0-bc79-84c2e79f336c for instance dcd3d415-addc-4607-844d-2ab1c2c9d16b on hv-cinder-maste Sep 30 09:54:04.147944 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-9eb54933-d790-4538-b996-872e2a1b69b3 req-32b27bbf-f863-4145-867b-e4880e7e4cc7 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046504 Sep 30 09:54:04.153944 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 431/852] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:54:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:54:04.193475 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:54:04.225718 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:04.602883 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-bd522156-c31c-410d-bef7-436387c0fb51 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "DELETE /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 204 len: 0 microversion: 2.1 time: 0.508423 Sep 30 09:54:04.605915 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 422/853] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:54:04 2022] DELETE /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 0 bytes in 510 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:54:04.615109 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:04.616542 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:04.625560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:04.626401 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:04.627635 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:04.628945 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:04.629616 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:04.630408 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:04.838973 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:04.841474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:04.872456 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:04.886261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:04.886261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:04.886261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:04.886261 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:04.886731 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:04.909624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:04.918208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b78d0a66-e3d6-4696-a4aa-a32c539d872c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.309687 Sep 30 09:54:04.920769 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 432/854] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:04 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:05.203633 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:05.237405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:05.243948 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:05.249668 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-481aa5fd-a860-4672-8ea9-249327f51a5a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.415089 Sep 30 09:54:05.250268 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 423/855] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:04 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:54:05.943201 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:05.943807 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:05.946171 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:05.950883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:05.951370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:05.951809 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:05.952497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:05.952951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:05.953378 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:06.237226 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:06.239522 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:06.245545 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3915c8f2-94a9-4085-8e24-bc6e71309a88 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.306301 Sep 30 09:54:06.246101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 433/856] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:05 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:06.279348 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:06.280084 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:06.287447 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:06.288015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:06.288527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:06.289297 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:06.290036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:06.290523 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:06.482614 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:06.494262 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:06.503591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:06.503591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d52cc2fc-0430-4a6f-a822-e77a5e85ae12 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.222391 Sep 30 09:54:06.503591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 424/857] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:06 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:07.337586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:07.338196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:07.345156 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:07.345880 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:07.347035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:07.352590 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:07.352590 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:07.352590 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:07.497354 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:07.503795 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ed900ab5-c53c-4651-8ed9-47d5e22a761e tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.170140 Sep 30 09:54:07.504252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 434/858] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:07 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:07.524145 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:07.524777 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:07.531498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:07.532606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:07.533046 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:07.533878 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:07.534864 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:07.535286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:07.672700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:07.680340 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:07.681833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:07.683055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-390ee7e2-4d80-42f5-b086-70793d5c4bab tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.162734 Sep 30 09:54:07.683651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 425/859] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:07 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:08.529915 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:08.530829 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:08.538381 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:08.539031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:08.539474 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:08.540170 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:08.540622 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:08.541045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:08.671057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:08.679214 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-65aeb558-af8b-4db9-8e43-9d433a95ee27 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1569 microversion: 2.1 time: 0.154294 Sep 30 09:54:08.679793 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 435/860] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:08 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1569 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:08.708628 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:08.709138 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:08.716001 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:08.716513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:08.716960 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:08.717657 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:08.718111 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:08.718669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:08.937494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:08.951213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:08.953543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:08.956568 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-46c94d6c-a4a3-4346-a36b-d8f1564adb36 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.251892 Sep 30 09:54:08.958436 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 426/861] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:08 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:09.446395 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-ad65c0f8-9c32-41b4-ab10-538a53d69e6f req-b2a7710e-ec81-46c6-b298-8da3feb1ef9b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dcd3d415-addc-4607-844d-2ab1c2c9d16b", "name": "network-vif-deleted", "tag": "803fcd8d-66b3-48a0-bc79-84c2e79f336c"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:54:09.456732 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ad65c0f8-9c32-41b4-ab10-538a53d69e6f req-b2a7710e-ec81-46c6-b298-8da3feb1ef9b service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:09.457227 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ad65c0f8-9c32-41b4-ab10-538a53d69e6f req-b2a7710e-ec81-46c6-b298-8da3feb1ef9b service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:09.457646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-ad65c0f8-9c32-41b4-ab10-538a53d69e6f req-b2a7710e-ec81-46c6-b298-8da3feb1ef9b service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:09.468746 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-ad65c0f8-9c32-41b4-ab10-538a53d69e6f req-b2a7710e-ec81-46c6-b298-8da3feb1ef9b service nova] Creating event network-vif-deleted:803fcd8d-66b3-48a0-bc79-84c2e79f336c for instance dcd3d415-addc-4607-844d-2ab1c2c9d16b on hv-cinder-maste Sep 30 09:54:09.473798 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-ad65c0f8-9c32-41b4-ab10-538a53d69e6f req-b2a7710e-ec81-46c6-b298-8da3feb1ef9b service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031327 Sep 30 09:54:09.474370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 436/862] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:54:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:54:09.701685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:09.705488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:09.711918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:09.712539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:09.712877 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:09.713566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:09.714009 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:09.714423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:09.791058 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:09.800884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7bae8715-446a-44f7-bac4-b733548210dd tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1220 microversion: 2.1 time: 0.101840 Sep 30 09:54:09.809512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 427/863] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:09 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1220 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:09.983902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:09.984689 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:09.991445 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:09.992124 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:09.993369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:09.993675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:09.994280 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:09.995023 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:10.146294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:10.146294 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:10.146557 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:10.146557 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2fdc3e9c-0735-4f1b-ad44-58dbd913ac81 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.165687 Sep 30 09:54:10.146557 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 437/864] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:09 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:10.823142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:10.823674 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:10.830073 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:10.830573 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:10.831635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:10.832365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:10.833006 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:10.833419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:10.862990 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:10.863469 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:10.863894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:10.864326 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lazy-loading 'fault' on Instance uuid dcd3d415-addc-4607-844d-2ab1c2c9d16b {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:54:10.906157 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20bee9f38eb14541a393037eaa389a44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1330031e214045bf9d45c73da1445255', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:10.914140 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1b8d04ce-505f-40d0-b1b4-5f8c55349ceb tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 200 len: 1225 microversion: 2.1 time: 0.094838 Sep 30 09:54:10.914715 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 428/865] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:10 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:11.170251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:11.170877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:11.177932 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:11.178427 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:11.178909 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:11.179615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:11.180208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:11.180637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:11.309589 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:11.318751 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:11.320226 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:11.321370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-504c0375-91a2-4bb8-a14a-fc5def39b7c3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.155023 Sep 30 09:54:11.321924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 438/866] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:11 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:54:11.937638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:11.938198 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [instance: dcd3d415-addc-4607-844d-2ab1c2c9d16b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:11.944499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:11.944982 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:11.945410 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:11.946097 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:11.946547 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:11.946993 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:11.960567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] HTTP exception thrown: Instance dcd3d415-addc-4607-844d-2ab1c2c9d16b could not be found. Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Request method failure captured: Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b HTTP/1.1 Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrt5AVw4biDItsDC1aERjs2O9a2p07tB7BllQj8qzZRh2N_leGUZr0cRp7PhyzLl-t0qudoaW5u4XJ2AWVIKz7GRWX_iJ2jw3pXGSJlHz0s7mNO5emeRKCdJpSHQQO7qVf7gy1SoC9DfkaMoLlsU5uHoBvXTUMIBBdbDiJhCdWtobmQ Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 1330031e214045bf9d45c73da1445255 Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-TestShelveInstance-2098923934 Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:54:11.962165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/1330031e214045bf9d45c73da1445255"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1330031e214045bf9d45c73da1445255"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1330031e214045bf9d45c73da1445255"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_1330031e214045bf9d45c73da1445255"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-TestShelveInstance-2098923934 Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 1330031e214045bf9d45c73da1445255 Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-TestShelveInstance-2098923934 Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-TestShelveInstance-2098923934-project Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 20bee9f38eb14541a393037eaa389a44 Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-TestShelveInstance-2098923934-project Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance dcd3d415-addc-4607-844d-2ab1c2c9d16b could not be found. Sep 30 09:54:11.963925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': 'dcd3d415-addc-4607-844d-2ab1c2c9d16b'} Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dcd3d415-addc-4607-844d-2ab1c2c9d16b could not be found. Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:54:11.964951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dcd3d415-addc-4607-844d-2ab1c2c9d16b could not be found. Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dcd3d415-addc-4607-844d-2ab1c2c9d16b could not be found. Sep 30 09:54:11.966566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:11.968087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Returning 404 to user: Instance dcd3d415-addc-4607-844d-2ab1c2c9d16b could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:54:11.968087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-dc9cf4e1-6258-412c-8ba9-e153afa38b23 tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b" status: 404 len: 111 microversion: 2.1 time: 0.029968 Sep 30 09:54:11.968087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 429/867] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:11 2022] GET /compute/v2.1/servers/dcd3d415-addc-4607-844d-2ab1c2c9d16b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:54:12.184500 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-67c3316c-a004-4400-9747-ea9c5a25c78c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] Calling method '.version_select at 0x7f72c0a393a0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:12.190584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR oslo.messaging._drivers.impl_rabbit [None req-67c3316c-a004-4400-9747-ea9c5a25c78c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [1936d0e6-5143-4334-af9a-793f5aa6a285] AMQP server on 10.222.0.39:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 30 09:54:12.346300 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:12.347051 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:12.353050 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:12.359394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:12.359394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:12.359394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:12.359394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:12.359854 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:12.486194 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:12.495160 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:12.497488 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:12.497783 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e5437007-e7c3-4456-bd31-23aad0ccb586 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.155375 Sep 30 09:54:12.498336 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 430/868] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:12 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:54:13.199315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO oslo.messaging._drivers.impl_rabbit [None req-67c3316c-a004-4400-9747-ea9c5a25c78c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] [1936d0e6-5143-4334-af9a-793f5aa6a285] Reconnected to AMQP server on 10.222.0.39:5672 via [amqp] client with port 51072. Sep 30 09:54:13.288045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-67c3316c-a004-4400-9747-ea9c5a25c78c tempest-TestShelveInstance-2098923934 tempest-TestShelveInstance-2098923934-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1312266241" status: 202 len: 0 microversion: 2.1 time: 1.107405 Sep 30 09:54:13.288671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 439/869] 10.222.0.39 () {58 vars in 1254 bytes} [Fri Sep 30 09:54:12 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1312266241 => generated 0 bytes in 1109 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:54:13.524378 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:13.524986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:13.531352 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:13.531832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:13.532259 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:13.532946 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:13.533396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:13.533813 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:13.669199 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:13.678251 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:13.679774 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:13.680920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-cf32bcd0-6b7a-48d7-b718-bbfb6a58f840 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.160577 Sep 30 09:54:13.681520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 431/870] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:13 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:14.708326 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:14.709308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:14.715388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:14.715978 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:14.716485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:14.717257 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:14.717784 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:14.718214 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:14.860831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:14.871082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:14.872614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:14.873804 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-59334e60-d150-46b3-9b40-13fc918edbac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.170306 Sep 30 09:54:14.874363 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 440/871] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:14 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:15.900891 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:15.902274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:15.913631 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:15.915712 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:15.916804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:15.917678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:15.918453 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:15.919065 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:15.919688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:16.153341 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:16.164830 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:16.166383 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:16.167639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d30faec3-c2a2-47b0-9f6b-d32c95a88ae5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.271829 Sep 30 09:54:16.179061 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 432/872] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:15 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:54:17.194732 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:17.196187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:17.202847 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:17.203337 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:17.205284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:17.205284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:17.205284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:17.205559 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:17.367091 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:17.380048 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:17.382224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:17.383790 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-88ae758e-cba4-4775-a5ea-4511cb98a9d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.194029 Sep 30 09:54:17.389940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 441/873] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:17 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:18.409077 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:18.409639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:18.416423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:18.416901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:18.417456 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:18.418140 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:18.418640 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:18.419047 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:18.554819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:18.564380 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:18.566503 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:18.571484 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b6ca9fb4-2ad9-4f42-9e2a-412bfc4a59b8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.162827 Sep 30 09:54:18.581714 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 433/874] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:18 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:19.466237 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-6104e6bd-e22d-40bd-83eb-b0c0c06c0be4 req-d4dff550-5fe1-439a-9adb-a1dc8d05caa4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dbe90c99-924b-4f03-85d5-4bfa0a6cd051", "name": "network-vif-plugged", "status": "completed", "tag": "869788d7-cd2f-4701-b6bd-a6f79a97b732"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:54:19.481344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6104e6bd-e22d-40bd-83eb-b0c0c06c0be4 req-d4dff550-5fe1-439a-9adb-a1dc8d05caa4 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:19.482083 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6104e6bd-e22d-40bd-83eb-b0c0c06c0be4 req-d4dff550-5fe1-439a-9adb-a1dc8d05caa4 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:19.482735 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-6104e6bd-e22d-40bd-83eb-b0c0c06c0be4 req-d4dff550-5fe1-439a-9adb-a1dc8d05caa4 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:19.497438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-6104e6bd-e22d-40bd-83eb-b0c0c06c0be4 req-d4dff550-5fe1-439a-9adb-a1dc8d05caa4 service nova] Creating event network-vif-plugged:869788d7-cd2f-4701-b6bd-a6f79a97b732 for instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 on hv-cinder-maste Sep 30 09:54:19.508384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-6104e6bd-e22d-40bd-83eb-b0c0c06c0be4 req-d4dff550-5fe1-439a-9adb-a1dc8d05caa4 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061984 Sep 30 09:54:19.508968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 442/875] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:54:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:54:19.605469 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:19.606280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:19.612832 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:19.613397 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:19.613900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:19.614688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:19.615203 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:19.615688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:19.826808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:19.839659 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:19.841474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:19.843264 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6da9e2a6-c442-47fa-bebd-3e6e0d2d15d7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.241766 Sep 30 09:54:19.859279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 434/876] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:19 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:20.870219 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:20.870946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:20.877987 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:20.878857 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:20.879295 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:20.880001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:20.880458 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:20.880878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:21.031649 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:21.041205 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:21.042679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:21.044097 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-65f99be8-fc73-472c-848a-9c06b3f15546 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.177894 Sep 30 09:54:21.044514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 443/877] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:20 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:22.071533 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:22.073466 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:22.082071 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:22.082669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:22.083233 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:22.084183 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:22.084952 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:22.085573 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:22.322142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:22.333492 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:22.336303 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:22.336303 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-62343d65-5a95-4d62-9a9f-c29cfd7fee80 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.270696 Sep 30 09:54:22.336656 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 435/878] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:22 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:23.362976 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:23.365270 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:23.370558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:23.371236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:23.371864 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:23.372755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:23.373383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:23.373811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:23.523416 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:23.532431 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:23.536158 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:23.537515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-54dc0b5d-11a6-4ad2-b8e4-6966a970305d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1808 microversion: 2.20 time: 0.180063 Sep 30 09:54:23.538019 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 444/879] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:23 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1808 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:54:24.564460 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:24.566287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:24.573885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:24.574632 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:24.575085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:24.575792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:24.576249 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:24.576676 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:24.711711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:24.720408 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:24.721921 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:24.723132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-358c7669-b090-4283-9131-d25400778673 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1795 microversion: 2.20 time: 0.162885 Sep 30 09:54:24.738630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 436/880] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:54:24 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1795 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:54:39.115970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:39.116523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:39.119541 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=24,nova_api:DELETE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:39.120329 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=53 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:39.124724 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:39.125677 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:39.125677 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:39.126332 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:39.126989 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:39.134885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:39.153031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:54:39.443192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c176e159-1d2a-41ba-a933-2d4a0be7416b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "DELETE /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 204 len: 0 microversion: 2.1 time: 0.330805 Sep 30 09:54:39.443929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 445/881] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:54:39 2022] DELETE /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:54:39.452002 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:39.452608 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:39.455541 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=9 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:39.457436 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:39.462019 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:39.462498 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:39.462955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:39.463644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:39.464088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:39.464636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:39.596724 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:39.605310 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1a5dc07b-3b8b-4671-b55b-4a977e6fb83d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1578 microversion: 2.1 time: 0.157149 Sep 30 09:54:39.606004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 437/882] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:39 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1578 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:40.630554 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:40.631292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:40.637712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:40.638196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:40.638652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:40.639352 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:40.639804 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:40.640307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:40.786426 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:40.794631 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-af9c39d8-7d82-471d-9e66-e202c8d64df4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1578 microversion: 2.1 time: 0.169130 Sep 30 09:54:40.795274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 446/883] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:40 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1578 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:41.818561 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:41.819202 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:41.827044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:41.827516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:41.827940 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:41.828630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:41.829107 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:41.829524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:41.974555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:41.983956 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1b9c575e-23d4-45cc-9056-67254a91afcf tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1578 microversion: 2.1 time: 0.170189 Sep 30 09:54:41.984908 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 438/884] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:41 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1578 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:42.407719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-4391c7f0-bdfe-4e7d-a640-038262db95b1 req-0704ddb2-4460-4748-82a2-21207e9a0d76 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bcd7505a-8f4d-4b0b-969e-776089b2e6d1", "name": "network-vif-deleted", "tag": "c69dcca2-dd34-464e-98ba-2697ae2f9628"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:54:42.424196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-4391c7f0-bdfe-4e7d-a640-038262db95b1 req-0704ddb2-4460-4748-82a2-21207e9a0d76 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:42.424196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-4391c7f0-bdfe-4e7d-a640-038262db95b1 req-0704ddb2-4460-4748-82a2-21207e9a0d76 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:42.424196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-4391c7f0-bdfe-4e7d-a640-038262db95b1 req-0704ddb2-4460-4748-82a2-21207e9a0d76 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:42.431691 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-4391c7f0-bdfe-4e7d-a640-038262db95b1 req-0704ddb2-4460-4748-82a2-21207e9a0d76 service nova] Creating event network-vif-deleted:c69dcca2-dd34-464e-98ba-2697ae2f9628 for instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 on hv-cinder-maste Sep 30 09:54:42.436751 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-4391c7f0-bdfe-4e7d-a640-038262db95b1 req-0704ddb2-4460-4748-82a2-21207e9a0d76 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032815 Sep 30 09:54:42.437424 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 447/885] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:54:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:54:43.009042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:43.009701 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:43.017083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:43.017562 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:43.018190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:43.019091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:43.019712 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:43.020301 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:43.092218 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:43.100720 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a560de90-9dfd-444e-bbc1-4ac4711144db tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1226 microversion: 2.1 time: 0.095686 Sep 30 09:54:43.101300 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 439/886] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:43 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:44.124809 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:44.125807 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:44.133516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:44.137757 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:44.137757 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:44.137757 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:44.137757 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:44.138513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:44.175462 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:44.175951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:44.176382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:44.176847 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lazy-loading 'fault' on Instance uuid bcd7505a-8f4d-4b0b-969e-776089b2e6d1 {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:54:44.219960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:44.227889 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c42ef688-021b-4755-a536-7a3cf045b8c7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 200 len: 1185 microversion: 2.1 time: 0.108312 Sep 30 09:54:44.228537 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 448/887] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:44 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 1185 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:45.254540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:45.255121 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:45.261481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:45.261957 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:45.262387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:45.263100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:45.263545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:45.263963 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:45.277603 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] HTTP exception thrown: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Request method failure captured: Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 HTTP/1.1 Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrwbJ3m7BfeNyh8xFop0ZrSTHHKK21Y7TdEWVYOY981ZqdFX0a9H7WcQ4DWGzk0TEbHy0wWxSWu5CzdUoC4SijJ_Vsyyw0tRnkBt6GRjmAtPzOP7lkvEE6VbxsqG8jjOLgxBhtOpmz_2iXVO_1w7lEfD2STZdTrOoIC2NMHTKeuq5Sw Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:54:45.279130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: c1df4fe7054840feac0a648ce82eea1a Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:54:45.280806 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': 'bcd7505a-8f4d-4b0b-969e-776089b2e6d1'} Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:54:45.281804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:54:45.283506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:54:45.284939 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Returning 404 to user: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:54:45.284939 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-06900ccd-78a9-4b12-afb1-6f70ddb3276e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 404 len: 111 microversion: 2.1 time: 0.030809 Sep 30 09:54:45.284939 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 440/888] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:45 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:54:45.375497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1733503552", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1635713456", "security_groups": [{"name": "tempest-secgroup-smoke-844905994"}], "block_device_mapping_v2": [{"uuid": "7bf1001d-9e49-498a-95db-ad274a5eff37", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3408b8b1-1a84-4fa0-8af6-da7419c2525d"}]}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:54:45.489104 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] validate_networks() for [('3408b8b1-1a84-4fa0-8af6-da7419c2525d', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:54:45.853055 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:54:45.853476 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:54:45.853990 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:54:45.854396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:54:45.854848 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:54:45.864675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:45.865238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:45.865672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:45.866388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:45.866862 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:45.867299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:45.884714 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Getting quotas for project f0d0c96bbb304882b1f2459361551a93. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:54:45.888693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Getting quotas for user c1df4fe7054840feac0a648ce82eea1a and project f0d0c96bbb304882b1f2459361551a93. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:54:45.944070 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:54:45.946181 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='7bf1001d-9e49-498a-95db-ad274a5eff37',volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:54:46.330370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4cd905bd-523b-41ce-bf54-7b639f59099e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.958680 Sep 30 09:54:46.334429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 449/889] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:54:45 2022] POST /compute/v2.1/servers => generated 405 bytes in 960 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:54:46.342701 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-49020f3c-726f-4bbb-982c-9237bf381018 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:46.343293 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-49020f3c-726f-4bbb-982c-9237bf381018 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:46.401497 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-49020f3c-726f-4bbb-982c-9237bf381018 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:46.410885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-49020f3c-726f-4bbb-982c-9237bf381018 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1095 microversion: 2.1 time: 0.072006 Sep 30 09:54:46.411448 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 441/890] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:46 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1095 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:47.433938 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:47.434564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:47.441198 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:47.441817 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:47.442254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:47.442975 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:47.443426 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:47.443847 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:47.503801 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:47.512227 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-958707b7-457b-456c-8085-8ee6ac12a7fd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1145 microversion: 2.1 time: 0.082662 Sep 30 09:54:47.513008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 450/891] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:47 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1145 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:48.535571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:48.536185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:48.542847 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:48.543335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:48.543775 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:48.544480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:48.544936 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:48.545366 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:48.612918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:48.623510 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-912b4be8-c439-4ba3-9bad-c605db523d07 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1201 microversion: 2.1 time: 0.092480 Sep 30 09:54:48.628116 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 442/892] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:48 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1201 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:49.650422 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:49.652238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:49.662461 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:49.662978 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:49.663413 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:49.664118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:49.664571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:49.664996 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:49.805130 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:49.813509 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-efead2cd-cccb-4826-a9d4-6b62ff28ac48 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1278 microversion: 2.1 time: 0.169921 Sep 30 09:54:49.814313 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 451/893] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:49 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1278 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:50.313675 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-0f9c8797-b407-4fb1-a7d3-4397b0f77497 req-f7624ad4-4759-4b51-a3c2-a5f0a49594e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3ee8e26b-68fe-4a79-a257-1d9456e4b22b", "tag": "3570f723-a6b6-479e-9662-f90c771a0449"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:54:50.324391 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0f9c8797-b407-4fb1-a7d3-4397b0f77497 req-f7624ad4-4759-4b51-a3c2-a5f0a49594e1 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:50.324863 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0f9c8797-b407-4fb1-a7d3-4397b0f77497 req-f7624ad4-4759-4b51-a3c2-a5f0a49594e1 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:50.325278 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0f9c8797-b407-4fb1-a7d3-4397b0f77497 req-f7624ad4-4759-4b51-a3c2-a5f0a49594e1 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:50.336780 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-0f9c8797-b407-4fb1-a7d3-4397b0f77497 req-f7624ad4-4759-4b51-a3c2-a5f0a49594e1 service nova] Creating event network-changed:3570f723-a6b6-479e-9662-f90c771a0449 for instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b on hv-cinder-maste Sep 30 09:54:50.341858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-0f9c8797-b407-4fb1-a7d3-4397b0f77497 req-f7624ad4-4759-4b51-a3c2-a5f0a49594e1 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033199 Sep 30 09:54:50.342445 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 443/894] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:54:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:54:50.838255 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:50.839325 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:50.845547 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:50.846026 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:50.846472 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:50.847801 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:50.848225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:50.848645 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:50.988643 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:50.996977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ec59b91b-4385-4f00-abd6-b73a623c7b52 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.162583 Sep 30 09:54:50.999719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 452/895] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:50 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:52.022772 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:52.023474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:52.029554 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:52.030034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:52.030465 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:52.031179 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:52.031624 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:52.032042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:52.169403 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:52.178106 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ed5056c1-5349-4732-84f6-40dbdf0ccba7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.160130 Sep 30 09:54:52.178693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 444/896] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:52 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:53.201441 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:53.202063 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:53.208732 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:53.213035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:53.213035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:53.213035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:53.213035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:53.213448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:53.342534 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:53.350729 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-128afe15-ee69-4def-a0ff-2edb025dcec7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.154003 Sep 30 09:54:53.351285 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 453/897] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:53 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:54.373980 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:54.374604 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:54.382560 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:54.383082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:54.386653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:54.386653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:54.386653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:54.386653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:54.525588 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:54.535407 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1911429d-463b-4048-9bf9-2233eb1c7899 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.163969 Sep 30 09:54:54.535407 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 445/898] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:54 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:55.555435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:55.556042 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:55.562099 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:55.562662 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:55.563028 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:55.563715 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:55.564244 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:55.564654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:55.748732 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:55.756831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e58533a6-8859-42e4-b2c9-7d7c2aceee31 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.205337 Sep 30 09:54:55.757397 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 454/899] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:55 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:56.780525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:56.781150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:56.783629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=2 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:56.788890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:56.789370 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:56.789833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:56.790519 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:56.790991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:56.796838 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:56.919223 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:56.927359 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a5184503-1617-4223-9c52-186d2632bcfd tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.150728 Sep 30 09:54:56.927931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 446/900] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:56 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:54:57.473651 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:57.474181 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:57.476083 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:57.476742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=6 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:57.481949 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:57.482444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:57.482910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:57.483606 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:57.484059 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:57.489578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:57.513193 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-529cc5a6-6071-4d1c-b354-3bf420dc5e91 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments/952937cf-c588-47d4-9ede-7fd97eda1772" status: 200 len: 194 microversion: 2.20 time: 0.044263 Sep 30 09:54:57.515763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 455/901] 10.222.0.39 () {60 vars in 1454 bytes} [Fri Sep 30 09:54:57 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments/952937cf-c588-47d4-9ede-7fd97eda1772 => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:54:57.581240 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:57.581736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:57.588192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:57.588665 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:57.589092 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:57.589777 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:57.590222 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:57.590667 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:57.668962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lazy-loading 'services' on Instance uuid dbe90c99-924b-4f03-85d5-4bfa0a6cd051 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:54:57.914099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-12d3f9d6-292a-4fd3-a425-6c87d0d03a2e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "DELETE /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments/952937cf-c588-47d4-9ede-7fd97eda1772" status: 202 len: 0 microversion: 2.20 time: 0.336614 Sep 30 09:54:57.914796 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 447/902] 10.222.0.39 () {60 vars in 1457 bytes} [Fri Sep 30 09:54:57 2022] DELETE /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments/952937cf-c588-47d4-9ede-7fd97eda1772 => generated 0 bytes in 338 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:54:57.923401 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:57.925627 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:57.932114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:57.932592 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:57.933046 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:57.933736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:57.934182 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:57.934632 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:57.950455 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:57.951183 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:57.958051 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:57.958539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:57.958986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:57.959731 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:57.960106 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:57.960517 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:57.965819 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-13346522-5a7a-4d9c-bdc2-73fcdce46528 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046596 Sep 30 09:54:57.966384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 456/903] 10.222.0.39 () {60 vars in 1343 bytes} [Fri Sep 30 09:54:57 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:54:58.099638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:58.107568 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-21b36524-5372-4323-89ac-6e040bbad79e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.161254 Sep 30 09:54:58.108348 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 448/904] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:57 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:54:58.978951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:58.981132 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:58.987804 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:58.988860 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:58.989353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:58.990060 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:58.990489 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:58.990929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:59.020486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5d7fb123-ad98-4a65-b015-4568c3d7bed9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046004 Sep 30 09:54:59.021035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 457/905] 10.222.0.39 () {60 vars in 1343 bytes} [Fri Sep 30 09:54:58 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:54:59.130104 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:54:59.130681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:54:59.136663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:59.137149 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:59.137583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:59.138272 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:54:59.138785 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:54:59.139214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:54:59.269898 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:54:59.277665 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d48f2824-6c81-4577-8d25-e3662389be6d tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.152293 Sep 30 09:54:59.278368 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 449/906] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:54:59 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:55:00.033442 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:00.036245 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:00.043623 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:00.044068 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:00.044497 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:00.045188 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:00.045632 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:00.046047 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:00.075482 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8740fade-3774-41d6-a224-ea32b91baba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.046081 Sep 30 09:55:00.075956 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 458/907] 10.222.0.39 () {60 vars in 1343 bytes} [Fri Sep 30 09:55:00 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:55:00.139085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:00.139606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:00.146552 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:00.147056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:00.147476 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:00.148153 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:00.148593 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:00.149004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:00.173697 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:55:00.300791 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:00.301360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:00.307145 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:00.307624 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:00.308088 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:00.308769 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:00.309216 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:00.311091 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:00.435064 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4ea23673-ef06-4c8d-81c3-c2b49e03f799 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "DELETE /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 204 len: 0 microversion: 2.20 time: 0.299547 Sep 30 09:55:00.435753 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 450/908] 10.222.0.39 () {60 vars in 1280 bytes} [Fri Sep 30 09:55:00 2022] DELETE /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 30 09:55:00.448136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:00.448302 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:00.448302 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:00.450979 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0f35918d-2cb9-4792-ad10-f9f364da4591 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.153975 Sep 30 09:55:00.451551 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:00.451693 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 459/909] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:55:00 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:55:00.452031 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:00.452467 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:00.453159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:00.453607 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:00.454025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:00.582796 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:00.590889 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:00.592310 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:00.593446 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-511e3233-2a95-43b3-8d24-1c4e6994c78f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.153475 Sep 30 09:55:00.593999 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 451/910] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:00 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:55:01.474163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:01.474823 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:01.481234 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:01.481719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:01.482155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:01.482868 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:01.483324 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:01.483750 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:01.619080 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:01.620248 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:01.628165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:01.630100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:01.630524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:01.631897 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:01.632641 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:01.633103 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:01.633527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:01.635367 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:01.640094 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3bdf13c4-1a45-4cf5-90d6-1dc3103f03ed tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.170478 Sep 30 09:55:01.640663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 460/911] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:55:01 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:55:01.770450 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-7ef581f7-bbed-480a-8d8a-dfaf13fc5012 req-70721bd2-ca52-4b39-8d09-42799a393ed1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3ee8e26b-68fe-4a79-a257-1d9456e4b22b", "name": "network-vif-plugged", "status": "completed", "tag": "3570f723-a6b6-479e-9662-f90c771a0449"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:55:01.776112 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:01.782960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7ef581f7-bbed-480a-8d8a-dfaf13fc5012 req-70721bd2-ca52-4b39-8d09-42799a393ed1 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:01.783436 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7ef581f7-bbed-480a-8d8a-dfaf13fc5012 req-70721bd2-ca52-4b39-8d09-42799a393ed1 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:01.783882 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-7ef581f7-bbed-480a-8d8a-dfaf13fc5012 req-70721bd2-ca52-4b39-8d09-42799a393ed1 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:01.785349 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:01.786895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:01.788051 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-efba8586-241a-4409-9d9e-02048e0122fb tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1723 microversion: 2.20 time: 0.173243 Sep 30 09:55:01.788722 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 452/912] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:01 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:55:01.795489 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-7ef581f7-bbed-480a-8d8a-dfaf13fc5012 req-70721bd2-ca52-4b39-8d09-42799a393ed1 service nova] Creating event network-vif-plugged:3570f723-a6b6-479e-9662-f90c771a0449 for instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b on hv-cinder-maste Sep 30 09:55:01.808374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-7ef581f7-bbed-480a-8d8a-dfaf13fc5012 req-70721bd2-ca52-4b39-8d09-42799a393ed1 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042459 Sep 30 09:55:01.808979 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 461/913] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:55:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:55:02.664005 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:02.664639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:02.671399 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:02.671872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:02.672291 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:02.673020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:02.673463 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:02.673879 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:02.692590 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:02.809824 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:02.816167 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:02.817045 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:02.818053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4ba74c31-3430-4668-b29d-fe992eb0b470 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1266 microversion: 2.1 time: 0.158629 Sep 30 09:55:02.826110 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 453/914] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:55:02 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:55:02.826281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:02.826281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:02.826281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:02.826281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:02.826646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:02.826646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:02.897993 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:02.906984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:02.908463 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:02.909614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-af0b4f27-b42d-47f9-bcc8-e775162a0c70 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1657 microversion: 2.20 time: 0.098469 Sep 30 09:55:02.910204 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 462/915] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:02 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1657 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:55:03.122194 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-54661c60-b8fd-47b6-bf2b-fff0832919b1 req-334d5029-cae4-4324-8322-221ce62bb04d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dbe90c99-924b-4f03-85d5-4bfa0a6cd051", "name": "network-vif-deleted", "tag": "869788d7-cd2f-4701-b6bd-a6f79a97b732"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:55:03.133103 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-54661c60-b8fd-47b6-bf2b-fff0832919b1 req-334d5029-cae4-4324-8322-221ce62bb04d service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:03.133513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-54661c60-b8fd-47b6-bf2b-fff0832919b1 req-334d5029-cae4-4324-8322-221ce62bb04d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:03.133937 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-54661c60-b8fd-47b6-bf2b-fff0832919b1 req-334d5029-cae4-4324-8322-221ce62bb04d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:03.145301 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-54661c60-b8fd-47b6-bf2b-fff0832919b1 req-334d5029-cae4-4324-8322-221ce62bb04d service nova] Creating event network-vif-deleted:869788d7-cd2f-4701-b6bd-a6f79a97b732 for instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 on hv-cinder-maste Sep 30 09:55:03.150125 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-54661c60-b8fd-47b6-bf2b-fff0832919b1 req-334d5029-cae4-4324-8322-221ce62bb04d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031619 Sep 30 09:55:03.150956 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 454/916] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:55:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:55:03.857350 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:03.857959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:03.869273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:03.869273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:03.869273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:03.869273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:03.869600 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:03.869600 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:03.937685 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:03.938270 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:03.945136 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:03.945911 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:03.946345 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:03.947094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:03.947545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:03.947968 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:03.986006 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:03.986006 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:03.986006 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:03.986312 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lazy-loading 'fault' on Instance uuid dbe90c99-924b-4f03-85d5-4bfa0a6cd051 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:55:04.039531 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:04.041133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:04.048558 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-84e2a068-a54b-4277-b238-16866bad63b9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1453 microversion: 2.1 time: 0.194999 Sep 30 09:55:04.049187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 463/917] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:55:03 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1453 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:55:04.049951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:04.052062 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:04.054886 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4c5b71bf-6842-4ad8-81b6-c05a8e1ad948 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 200 len: 1371 microversion: 2.20 time: 0.121288 Sep 30 09:55:04.054886 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 455/918] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:03 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 1371 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:55:04.071564 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:04.072314 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:04.078578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:04.079109 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:04.080644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:04.080644 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:04.080874 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:04.081255 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:04.228290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:04.236003 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7bf1986c-e003-410c-8c2d-9984c0422924 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1453 microversion: 2.1 time: 0.168603 Sep 30 09:55:04.236571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 464/919] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:55:04 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1453 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:55:05.078332 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:05.078938 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:05.085438 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:05.085922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:05.086350 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:05.087075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:05.087526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:05.087945 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:05.248462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] HTTP exception thrown: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Request method failure captured: Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 HTTP/1.1 Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrw-SBto1FeGFMxbdtMei32cZ7--DPKSAV2_AfLxBMLTGswXrtFtfcO2kIRYC2gSxbY9nedY0N1c3gCVrhIn0pqN4B2bnce3vgn6MluzHM9gAFz2QnvvHxybzWs0PFbKAPJ43ncFan3dI767urivXMaYwI_5gLRsRBe2eFIgKoQtVZ0 Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Openstack-Nova-Api-Version: 2.20 Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:55:05.250120 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 21d5e60badb846fb840a65f443728a9b Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:55:05.251930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': 'dbe90c99-924b-4f03-85d5-4bfa0a6cd051'} Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:55:05.257124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:55:05.258782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:55:05.260273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Returning 404 to user: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:55:05.260273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0a80e3a0-7ea1-48c6-8d5a-7ba8fb15c582 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 404 len: 111 microversion: 2.20 time: 0.177154 Sep 30 09:55:05.260273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 456/920] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:05 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 111 bytes in 178 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 30 09:55:05.261744 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-61c611b8-e076-4d31-9793-460a77e89524 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '.version_select at 0x7f72c0a99f70>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:05.329723 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-61c611b8-e076-4d31-9793-460a77e89524 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1043075220" status: 204 len: 0 microversion: 2.20 time: 0.071779 Sep 30 09:55:05.330448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 465/921] 10.222.0.39 () {60 vars in 1262 bytes} [Fri Sep 30 09:55:05 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1043075220 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 30 09:55:06.255065 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-af09ef33-19ef-4795-b5e3-4956345408ac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'create', calling method: .version_select at 0x7f72c0a40c10>, body: {"keypair": {"name": "tempest-keypair-1715098926", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFJr3Arn4hHGGw1L1EZqglDJUxnkkBepAsFgC3gTU2xJvPs3iuOFPH9aeWYKoIPQydd8nbWnCN8vSXnxJzg88UTZ3afVG3Exe4vU3gAhVBuL05e8/cQaDJjp8p6ek06reQ=="}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:55:06.278434 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-0e2e8be1-88db-44f5-ba73-c0cc9fa5327b req-d58d59d8-4e71-47f3-84b9-4f9aa626ea48 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3ee8e26b-68fe-4a79-a257-1d9456e4b22b", "tag": "3570f723-a6b6-479e-9662-f90c771a0449"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:55:06.283292 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-af09ef33-19ef-4795-b5e3-4956345408ac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Getting quotas for project 1142014cc9284cf1833e5bfbcdb41599. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:55:06.289310 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-af09ef33-19ef-4795-b5e3-4956345408ac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Getting quotas for user 21d5e60badb846fb840a65f443728a9b and project 1142014cc9284cf1833e5bfbcdb41599. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:55:06.293169 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0e2e8be1-88db-44f5-ba73-c0cc9fa5327b req-d58d59d8-4e71-47f3-84b9-4f9aa626ea48 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:06.293754 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0e2e8be1-88db-44f5-ba73-c0cc9fa5327b req-d58d59d8-4e71-47f3-84b9-4f9aa626ea48 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:06.294278 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-0e2e8be1-88db-44f5-ba73-c0cc9fa5327b req-d58d59d8-4e71-47f3-84b9-4f9aa626ea48 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:06.307134 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-0e2e8be1-88db-44f5-ba73-c0cc9fa5327b req-d58d59d8-4e71-47f3-84b9-4f9aa626ea48 service nova] Creating event network-changed:3570f723-a6b6-479e-9662-f90c771a0449 for instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b on hv-cinder-maste Sep 30 09:55:06.312278 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-0e2e8be1-88db-44f5-ba73-c0cc9fa5327b req-d58d59d8-4e71-47f3-84b9-4f9aa626ea48 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038055 Sep 30 09:55:06.312988 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 466/922] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:55:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:55:06.392588 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-af09ef33-19ef-4795-b5e3-4956345408ac tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.146445 Sep 30 09:55:06.393637 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 457/923] 10.222.0.39 () {62 vars in 1200 bytes} [Fri Sep 30 09:55:06 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 149 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 30 09:55:08.690368 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1882193074", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6555801d-cf6b-484f-a48c-039d4395e1ca"}], "security_groups": [{"name": "tempest-securitygroup--539401786"}], "key_name": "tempest-keypair-1715098926", "user_data": "}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:55:08.796963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] validate_networks() for [('6555801d-cf6b-484f-a48c-039d4395e1ca', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:55:09.160811 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:55:09.161236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:55:09.161711 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:55:09.162153 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:55:09.162566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:55:09.171530 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:09.172015 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:09.172453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:09.173185 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:09.173649 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:09.174078 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:09.190695 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Getting quotas for project 1142014cc9284cf1833e5bfbcdb41599. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:55:09.195268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Getting quotas for user 21d5e60badb846fb840a65f443728a9b and project 1142014cc9284cf1833e5bfbcdb41599. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:55:09.206440 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:55:09.209531 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:55:09.347836 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2cef24d7-c9a9-4de4-8f01-f619031f2954 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.662730 Sep 30 09:55:09.348437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 467/924] 10.222.0.39 () {62 vars in 1188 bytes} [Fri Sep 30 09:55:08 2022] POST /compute/v2.1/servers => generated 401 bytes in 664 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 30 09:55:09.360392 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2594f64d-8b3f-4e23-89b2-a331988ed7c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:09.361074 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2594f64d-8b3f-4e23-89b2-a331988ed7c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:09.415101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2594f64d-8b3f-4e23-89b2-a331988ed7c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:09.422403 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2594f64d-8b3f-4e23-89b2-a331988ed7c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:09.423873 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2594f64d-8b3f-4e23-89b2-a331988ed7c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:09.424961 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2594f64d-8b3f-4e23-89b2-a331988ed7c5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1282 microversion: 2.20 time: 0.069330 Sep 30 09:55:09.426381 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 458/925] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:09 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1282 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:10.451144 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:10.451750 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:10.453743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=81,nova_api:UPDATE=1,nova_api:INSERT=6,nova_api:DELETE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:10.459830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:10.463283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:10.463283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:10.463283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:10.463283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:10.463672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:10.524084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:10.532114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:10.533570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:10.534730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-6f89e357-18d8-470a-bd18-80e410ea84b2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1278 microversion: 2.20 time: 0.088037 Sep 30 09:55:10.535898 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 468/926] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:10 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1278 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:11.560635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:11.561232 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:11.563803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=65,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:11.568513 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:11.568974 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:11.569400 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:11.570096 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:11.570524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:11.570970 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:11.633070 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:11.642280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:11.643874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:11.644969 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4e2b83bc-3225-41f8-af52-71140a293d63 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1340 microversion: 2.20 time: 0.088325 Sep 30 09:55:11.645520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 459/927] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:11 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1340 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:12.671282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:12.671919 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:12.679103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:12.680169 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:12.680169 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:12.680675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:12.681129 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:12.681552 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:12.812741 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:12.821231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:12.822702 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:12.823816 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7a037f20-605f-4149-8b3f-c7c0c5e00a5d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1407 microversion: 2.20 time: 0.156895 Sep 30 09:55:12.824369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 469/928] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:12 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1407 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:13.073669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-2f4923a5-8083-4837-bab7-4f69a484a97d req-901a166c-a45e-4e69-9c7b-cdd21fc287f2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "299c80c2-f890-43f2-9e42-86f4939c18df", "tag": "5fe64bf6-a448-427d-8a92-4b647a91c107"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:55:13.084556 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-2f4923a5-8083-4837-bab7-4f69a484a97d req-901a166c-a45e-4e69-9c7b-cdd21fc287f2 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:13.085034 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-2f4923a5-8083-4837-bab7-4f69a484a97d req-901a166c-a45e-4e69-9c7b-cdd21fc287f2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:13.085464 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-2f4923a5-8083-4837-bab7-4f69a484a97d req-901a166c-a45e-4e69-9c7b-cdd21fc287f2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:13.096252 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-2f4923a5-8083-4837-bab7-4f69a484a97d req-901a166c-a45e-4e69-9c7b-cdd21fc287f2 service nova] Creating event network-changed:5fe64bf6-a448-427d-8a92-4b647a91c107 for instance 299c80c2-f890-43f2-9e42-86f4939c18df on hv-cinder-maste Sep 30 09:55:13.101911 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-2f4923a5-8083-4837-bab7-4f69a484a97d req-901a166c-a45e-4e69-9c7b-cdd21fc287f2 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033133 Sep 30 09:55:13.102499 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 460/929] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:55:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:55:13.848632 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:13.849240 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:13.855105 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:13.855540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:13.855978 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:13.856660 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:13.857136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:13.857545 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:13.982928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:13.991495 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:13.992867 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:13.994050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8868d0a4-c4aa-4ee9-bb59-dc03ad2219f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1407 microversion: 2.20 time: 0.149505 Sep 30 09:55:13.994612 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 470/930] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:13 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1407 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:15.020996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:15.021687 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:15.028158 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:15.028632 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:15.029058 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:15.029739 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:15.030188 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:15.030626 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:15.158950 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:15.166971 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:15.168401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:15.169485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8a91e8d7-d0a1-4942-b60c-46c8457c3925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1407 microversion: 2.20 time: 0.153718 Sep 30 09:55:15.170032 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 461/931] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:15 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1407 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:16.194342 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:16.194974 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:16.201472 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:16.201948 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:16.202375 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:16.203092 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:16.203543 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:16.203960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:16.329584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:16.338079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:16.339555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:16.340645 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-db63910c-da5b-47c9-b1db-5280e01e1ee5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1407 microversion: 2.20 time: 0.150268 Sep 30 09:55:16.341205 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 471/932] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:16 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1407 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:17.366753 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:17.367350 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:17.373691 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:17.374166 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:17.374608 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:17.375406 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:17.375850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:17.376262 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:17.505369 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:17.515107 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:17.515107 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:17.516122 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2c4565dc-00fa-4463-be0f-c67fee79194d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1407 microversion: 2.20 time: 0.154325 Sep 30 09:55:17.516672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 462/933] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:17 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1407 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:18.541443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:18.542049 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:18.548552 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:18.549035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:18.549468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:18.550175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:18.551129 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:18.551553 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:18.682103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:18.687114 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:18.688560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:18.689654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4c00355a-8a83-4c96-be2f-db0f21c24e47 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1407 microversion: 2.20 time: 0.152152 Sep 30 09:55:18.690207 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 472/934] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:18 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1407 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:19.715164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:19.715767 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:19.718186 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=6 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:19.722994 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:19.723420 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:19.723853 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:19.724551 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:19.725000 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:19.725421 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:19.856234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:19.863242 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-2df16dd8-ac4a-4ad1-92f9-f3a23f819340 req-40096933-0864-4e73-8fb6-308df61cd405 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "299c80c2-f890-43f2-9e42-86f4939c18df", "name": "network-vif-plugged", "status": "completed", "tag": "5fe64bf6-a448-427d-8a92-4b647a91c107"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:55:19.868696 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:19.869409 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=6 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:19.870099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:19.871713 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:19.872851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f4768901-b8d7-410b-901c-4b37000de520 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1407 microversion: 2.20 time: 0.161674 Sep 30 09:55:19.873407 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 463/935] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:19 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1407 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:19.875291 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-2df16dd8-ac4a-4ad1-92f9-f3a23f819340 req-40096933-0864-4e73-8fb6-308df61cd405 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:19.875765 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-2df16dd8-ac4a-4ad1-92f9-f3a23f819340 req-40096933-0864-4e73-8fb6-308df61cd405 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:19.876193 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-2df16dd8-ac4a-4ad1-92f9-f3a23f819340 req-40096933-0864-4e73-8fb6-308df61cd405 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:19.888032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-2df16dd8-ac4a-4ad1-92f9-f3a23f819340 req-40096933-0864-4e73-8fb6-308df61cd405 service nova] Creating event network-vif-plugged:5fe64bf6-a448-427d-8a92-4b647a91c107 for instance 299c80c2-f890-43f2-9e42-86f4939c18df on hv-cinder-maste Sep 30 09:55:19.892875 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-2df16dd8-ac4a-4ad1-92f9-f3a23f819340 req-40096933-0864-4e73-8fb6-308df61cd405 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033558 Sep 30 09:55:19.893443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 473/936] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:55:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:55:20.897872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:20.898428 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:20.904776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:20.905275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:20.909234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:20.909234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:20.909234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:20.909234 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:21.039704 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:21.047657 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:21.049100 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:21.050248 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9c95079f-941b-4107-9fce-b1c00ac1aca1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1407 microversion: 2.20 time: 0.156932 Sep 30 09:55:21.050846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 464/937] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:20 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1407 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:22.077079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:22.077769 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:22.085211 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:22.085688 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:22.086121 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:22.086834 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:22.087281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:22.087700 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:22.217024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:22.225752 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:22.227249 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:55:22.228710 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c01ba52f-cc7c-40e5-aa67-6e16290d89bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1599 microversion: 2.20 time: 0.156878 Sep 30 09:55:22.229297 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 474/938] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:55:22 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1599 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:55:22.252850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:22.253346 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:22.259966 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:22.260440 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:22.260872 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:22.261550 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:22.261991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:22.262406 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:22.336505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0e2fc10d-2dc2-4da6-8ead-8374a50c2cc1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.087474 Sep 30 09:55:22.337127 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 465/939] 10.222.0.39 () {60 vars in 1316 bytes} [Fri Sep 30 09:55:22 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/os-interface => generated 284 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:55:23.273527 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-d7976dc2-0bcf-4a92-a4a7-455cefe4e47b req-8ec75f6d-62d2-4dab-a1cc-80311f7d0470 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "299c80c2-f890-43f2-9e42-86f4939c18df", "tag": "5fe64bf6-a448-427d-8a92-4b647a91c107"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:55:23.283707 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-d7976dc2-0bcf-4a92-a4a7-455cefe4e47b req-8ec75f6d-62d2-4dab-a1cc-80311f7d0470 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:23.284384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-d7976dc2-0bcf-4a92-a4a7-455cefe4e47b req-8ec75f6d-62d2-4dab-a1cc-80311f7d0470 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:23.285383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-d7976dc2-0bcf-4a92-a4a7-455cefe4e47b req-8ec75f6d-62d2-4dab-a1cc-80311f7d0470 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:23.296736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-d7976dc2-0bcf-4a92-a4a7-455cefe4e47b req-8ec75f6d-62d2-4dab-a1cc-80311f7d0470 service nova] Creating event network-changed:5fe64bf6-a448-427d-8a92-4b647a91c107 for instance 299c80c2-f890-43f2-9e42-86f4939c18df on hv-cinder-maste Sep 30 09:55:23.306528 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-d7976dc2-0bcf-4a92-a4a7-455cefe4e47b req-8ec75f6d-62d2-4dab-a1cc-80311f7d0470 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032911 Sep 30 09:55:23.306528 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 475/940] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:55:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:55:59.201331 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=13 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:59.202094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=50 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:59.206053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:55:59.206541 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:55:59.212967 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:59.213457 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:59.216916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:59.216916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:55:59.216916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:55:59.216916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:55:59.239875 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ac911adc-7500-468f-b6e9-784d458b4041 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/ips" status: 200 len: 151 microversion: 2.20 time: 0.040036 Sep 30 09:55:59.245470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 466/941] 10.222.0.39 () {60 vars in 1289 bytes} [Fri Sep 30 09:55:59 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/ips => generated 151 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:56:01.423470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:56:01.424123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:01.426414 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=15 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:56:01.427100 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=64 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:56:01.432606 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:01.433090 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:01.433519 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:01.434210 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:01.434709 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:01.435134 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:01.457214 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lazy-loading 'flavor' on Instance uuid 299c80c2-f890-43f2-9e42-86f4939c18df {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:56:02.038958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-65a2c627-af93-464a-907e-86592daa588a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/action" status: 202 len: 0 microversion: 2.20 time: 0.619871 Sep 30 09:56:02.039585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 476/942] 10.222.0.39 () {62 vars in 1319 bytes} [Fri Sep 30 09:56:01 2022] POST /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/action => generated 0 bytes in 622 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:56:02.047968 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:02.048578 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:02.055629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:02.056118 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:02.056550 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:02.057250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:02.057700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:02.058124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:02.290716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:02.298543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:02.299993 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:02.301121 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e1d2e240-c279-4b8d-b71c-d0ec96560a93 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.257193 Sep 30 09:56:02.301670 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 467/943] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:02 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:56:03.327677 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:03.328376 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:03.334746 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:03.335232 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:03.335667 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:03.336956 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:03.337410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:03.337832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:03.464180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:03.472197 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:03.473636 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:03.474827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-83a46d83-04e0-4fcf-9b94-f01476e36c6b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.152013 Sep 30 09:56:03.475377 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 477/944] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:03 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:04.500402 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:04.501006 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:04.507576 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:04.508068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:04.508494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:04.509170 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:04.509612 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:04.510024 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:04.636102 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:04.644062 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:04.645492 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:04.646648 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c95e9992-3ce7-4aa3-b1ba-1fd441788b6c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.150422 Sep 30 09:56:04.647201 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 468/945] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:04 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:05.673340 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:05.673947 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:05.680963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:05.681428 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:05.681861 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:05.682548 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:05.683034 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:05.683485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:05.812885 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:05.820959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:05.822396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:05.823577 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4420c015-899f-40df-a3a3-e499c0a3f925 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.154236 Sep 30 09:56:05.824134 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 478/946] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:05 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:06.848739 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:06.849365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:06.855672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:06.856157 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:06.856585 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:06.857270 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:06.857744 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:06.858166 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:06.983214 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:06.991091 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:06.993006 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:06.994147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2635e39f-11d6-4a24-b5fb-8dc385bb4bd1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.149410 Sep 30 09:56:06.994760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 469/947] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:06 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:08.020726 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:08.021343 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:08.028403 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:08.028890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:08.029328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:08.030026 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:08.030486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:08.030943 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:08.165059 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:08.173126 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:08.174555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:08.175726 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9dc5d867-0d9a-473a-b766-70750874fe07 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.158905 Sep 30 09:56:08.176267 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 479/948] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:08 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:09.202509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:09.203143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:09.209543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:09.210019 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:09.210449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:09.211162 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:09.211608 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:09.212025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:09.336376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:09.344320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:09.345782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:09.346959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-98d92e05-c886-4682-a08b-50dba030fdc0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.148411 Sep 30 09:56:09.347511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 470/949] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:09 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:10.372615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:10.373215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:10.379680 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:10.380159 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:10.380839 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:10.381516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:10.381962 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:10.382381 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:10.506425 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:10.515014 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:10.516509 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:10.517663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3668fb01-1265-452b-a386-51b3f6941381 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.148962 Sep 30 09:56:10.518214 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 480/950] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:10 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:11.543066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:11.543696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:11.550063 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:11.550544 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:11.551007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:11.551690 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:11.552143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:11.552593 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:11.682064 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:11.690090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:11.691544 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:11.692681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-62a0c04e-0e59-4977-ab17-59e7063e4a8e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.154109 Sep 30 09:56:11.693231 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 471/951] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:11 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:12.719137 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:12.719747 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:12.727005 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:12.727498 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:12.727932 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:12.728620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:12.729077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:12.729501 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:12.854374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:12.862349 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:12.863832 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:12.864995 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-306ee735-dc8c-46ae-b0eb-880c8ac0ee2d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.149859 Sep 30 09:56:12.865551 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 481/952] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:12 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:13.890347 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:13.890984 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:13.897306 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:13.901068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:13.901068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:13.901068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:13.901068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:13.901470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:14.028328 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:14.036343 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:14.037764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:14.039494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-30529af9-bdd4-4384-88ee-dbfdb8c3a762 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.152478 Sep 30 09:56:14.040048 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 472/953] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:13 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:15.065556 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:15.066162 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:15.072776 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:15.073256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:15.073684 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:15.074365 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:15.074843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:15.075267 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:15.201578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:15.209743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:15.211282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:15.212423 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3ddfb489-daf4-45a1-9fed-6a2d49e9c8bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.151043 Sep 30 09:56:15.213055 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 482/954] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:15 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:16.237245 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:16.237849 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:16.244240 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:16.244721 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:16.245147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:16.245835 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:16.246286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:16.246735 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:16.375907 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:16.384646 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:16.386082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:16.387254 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-fddf32aa-91d5-4a4b-ad89-a1164b4c0223 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.153940 Sep 30 09:56:16.387804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 473/955] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:16 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:17.413077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:17.413734 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:17.420743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:17.421233 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:17.421672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:17.422366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:17.422861 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:17.423327 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:17.547720 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:17.555673 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:17.557161 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:17.558314 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-6049657a-5757-46c7-b25b-1dd1a47f88bf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.149200 Sep 30 09:56:17.558900 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 483/956] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:17 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:18.584056 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:18.584663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:18.591260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:18.591738 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:18.592168 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:18.592850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:18.593298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:18.593713 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:18.717833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:18.725723 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:18.727837 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:18.728985 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-52b16d23-b171-4d5d-a39a-2909b68c3ccf tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1744 microversion: 2.20 time: 0.148843 Sep 30 09:56:18.729542 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 474/957] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:18 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1744 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:19.754326 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:19.754958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:19.762621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:19.763105 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:19.763540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:19.764227 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:19.764674 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:19.765089 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:19.901783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:19.901783 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:19.902056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:19.902585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-625a6dfb-6abe-496e-8402-81d609bbcb31 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1744 microversion: 2.20 time: 0.152060 Sep 30 09:56:19.903157 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 484/958] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:19 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1744 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:20.928260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:20.928867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:20.936409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:20.937159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:20.937612 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:20.938370 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:20.938852 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:20.939333 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:21.072382 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:21.080501 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:21.081941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:21.083156 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f232cbe9-49d2-43cc-b906-80fec4d2a834 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.158828 Sep 30 09:56:21.083715 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 475/959] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:20 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:22.091283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-423c0a4c-aec8-4fb0-a255-e7791d854e4b req-aa2226f9-25bb-4934-8867-a42d29302f62 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "299c80c2-f890-43f2-9e42-86f4939c18df", "name": "network-vif-plugged", "status": "completed", "tag": "5fe64bf6-a448-427d-8a92-4b647a91c107"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:56:22.103568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-423c0a4c-aec8-4fb0-a255-e7791d854e4b req-aa2226f9-25bb-4934-8867-a42d29302f62 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:22.104337 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-423c0a4c-aec8-4fb0-a255-e7791d854e4b req-aa2226f9-25bb-4934-8867-a42d29302f62 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:22.107061 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-423c0a4c-aec8-4fb0-a255-e7791d854e4b req-aa2226f9-25bb-4934-8867-a42d29302f62 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:22.110173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:22.111953 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:22.120026 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:22.125518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:22.125518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:22.125518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:22.125518 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:22.125893 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:22.125977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-423c0a4c-aec8-4fb0-a255-e7791d854e4b req-aa2226f9-25bb-4934-8867-a42d29302f62 service nova] Creating event network-vif-plugged:5fe64bf6-a448-427d-8a92-4b647a91c107 for instance 299c80c2-f890-43f2-9e42-86f4939c18df on hv-cinder-maste Sep 30 09:56:22.133886 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-423c0a4c-aec8-4fb0-a255-e7791d854e4b req-aa2226f9-25bb-4934-8867-a42d29302f62 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046742 Sep 30 09:56:22.134928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 485/960] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:56:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:56:22.284798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:22.295443 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:22.296924 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:22.298088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1478f8e1-8a0b-4600-9b4c-b5ce117a9b67 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.192045 Sep 30 09:56:22.298669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 476/961] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:22 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:23.326681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:23.329268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:23.335654 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:23.336135 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:23.336566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:23.337296 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:23.337750 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:23.338168 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:23.502663 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:23.511984 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:23.513653 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:23.514968 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-be966a3a-d487-4f4a-bc3c-c7bae5b456b1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.194642 Sep 30 09:56:23.515786 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 486/962] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:23 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:24.541195 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:24.541784 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:24.548473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:24.549190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:24.549805 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:24.550708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:24.551328 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:24.551918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:24.695656 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:24.709692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:24.711155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:24.712837 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-fa2ef79a-dfcb-4593-8be7-9c439c0b1494 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.175600 Sep 30 09:56:24.713574 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 477/963] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:24 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:25.741540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:25.742287 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:25.751020 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:25.751545 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:25.752027 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:25.752737 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:25.753195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:25.753619 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:25.906110 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:25.918393 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:25.927393 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:25.928710 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-38a9ed2e-d2e2-4aa2-a760-57911d57b68c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.191983 Sep 30 09:56:25.929344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 487/964] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:25 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:26.956385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:26.957001 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:26.964908 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:26.965442 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:26.965998 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:26.966920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:26.968021 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:26.968511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:27.117504 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:27.126414 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:27.127892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:27.129079 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0c2593a6-ad40-4019-9e2a-9894c1d3eb56 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.177516 Sep 30 09:56:27.129635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 478/965] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:26 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:28.155282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:28.155883 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:28.163668 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:28.164163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:28.164594 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:28.165812 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:28.166261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:28.166711 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:28.317203 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:28.326079 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:28.327655 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:28.328826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fbaf70aa-f910-4ce5-8366-10117f3ac712 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.177454 Sep 30 09:56:28.331368 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 488/966] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:28 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:29.356951 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:29.357591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:29.361008 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=27 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:56:29.364340 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:29.365063 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:29.365514 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:29.366185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:29.366645 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:29.367065 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:29.511480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:29.524535 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:29.525988 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:29.527191 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c4af7c71-890f-4a8f-835a-d8f9c94e44be tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.176361 Sep 30 09:56:29.527900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 479/967] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:29 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:30.559843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:30.560578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:30.566830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:30.567320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:30.567756 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:30.568571 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:30.569030 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:30.569457 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:30.706761 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:30.715778 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:30.717277 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:30.718506 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-6eb0d3f1-83cb-4bf3-80ad-5aebf259fbc9 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.164915 Sep 30 09:56:30.719224 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 489/968] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:30 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:31.744364 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:31.744971 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:31.751676 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:31.752159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:31.752594 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:31.753316 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:31.753765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:31.754185 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:31.888090 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:31.896297 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:31.898160 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:31.899611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ae399fa9-94eb-47aa-a65a-f9d122136174 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1739 microversion: 2.20 time: 0.158376 Sep 30 09:56:31.900171 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 480/969] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:31 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1739 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:32.924935 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:32.925559 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:32.927965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=29 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:56:32.932702 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:32.933175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:32.933603 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:32.934283 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:32.934770 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:32.935196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:32.937688 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:56:33.061039 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:33.068948 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:33.070377 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:33.071572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-a054d323-33b9-4ee0-b1e2-a2be35116507 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1721 microversion: 2.20 time: 0.150625 Sep 30 09:56:33.072123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 490/970] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:32 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:34.097173 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:34.097773 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:34.104681 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:34.105144 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:34.105571 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:34.106249 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:34.106759 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:34.107177 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:34.110961 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=89 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:56:34.246325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:34.254625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:34.256325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:34.257468 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-037e4e47-c1d0-4107-a5a2-c5704c8de19b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1721 microversion: 2.20 time: 0.164210 Sep 30 09:56:34.258015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 481/971] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:34 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1721 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:56:35.282666 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:35.283282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:35.289838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:35.290328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:35.290803 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:35.291502 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:35.291961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:35.292394 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:35.418528 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:35.426538 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:35.428031 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:35.429182 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-47edc430-8c8c-4a15-ac9a-68ba386ae384 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1664 microversion: 2.20 time: 0.150494 Sep 30 09:56:35.429738 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 491/972] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:35 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1664 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:35.461988 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9cdde3da-2dd5-4099-9434-23f253ae4fc5"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:56:35.464239 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:35.470555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:35.471059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:35.471483 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:35.472154 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:35.472640 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:35.473055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:36.180962 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-9e44b0a0-c8e6-49cf-8c72-1420885a1662 req-e12d43e3-f83f-4016-9850-559d9747278d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "299c80c2-f890-43f2-9e42-86f4939c18df", "name": "network-vif-unplugged", "status": "completed", "tag": "5fe64bf6-a448-427d-8a92-4b647a91c107"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:56:36.190393 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-9e44b0a0-c8e6-49cf-8c72-1420885a1662 req-e12d43e3-f83f-4016-9850-559d9747278d service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:36.192438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-9e44b0a0-c8e6-49cf-8c72-1420885a1662 req-e12d43e3-f83f-4016-9850-559d9747278d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:36.192438 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-9e44b0a0-c8e6-49cf-8c72-1420885a1662 req-e12d43e3-f83f-4016-9850-559d9747278d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:36.202014 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.compute.server_external_events [req-9e44b0a0-c8e6-49cf-8c72-1420885a1662 req-e12d43e3-f83f-4016-9850-559d9747278d service nova] Unable to find a host for instance 299c80c2-f890-43f2-9e42-86f4939c18df. Dropping event network-vif-unplugged {{(pid=127456) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 30 09:56:36.202916 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-9e44b0a0-c8e6-49cf-8c72-1420885a1662 req-e12d43e3-f83f-4016-9850-559d9747278d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.026554 Sep 30 09:56:36.203549 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 492/973] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:56:36 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 28 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Sep 30 09:56:36.503218 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7aac5788-58e3-4148-b744-8aa35a760bc4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.045322 Sep 30 09:56:36.504559 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 482/974] 10.222.0.39 () {62 vars in 1364 bytes} [Fri Sep 30 09:56:35 2022] POST /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/os-volume_attachments => generated 188 bytes in 1047 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 30 09:56:36.628618 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:36.629104 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:36.636377 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:36.636865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:36.637356 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:36.637998 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:36.638453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:36.638913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:37.193621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-42fdf7de-da9b-4126-932d-7d52b75b3ba7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "DELETE /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/os-volume_attachments/9cdde3da-2dd5-4099-9434-23f253ae4fc5" status: 202 len: 0 microversion: 2.20 time: 0.569063 Sep 30 09:56:37.194146 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 493/975] 10.222.0.39 () {60 vars in 1457 bytes} [Fri Sep 30 09:56:36 2022] DELETE /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/os-volume_attachments/9cdde3da-2dd5-4099-9434-23f253ae4fc5 => generated 0 bytes in 570 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:56:37.256141 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:56:37.256736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:37.263227 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:37.263713 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:37.264142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:37.264824 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:37.265269 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:37.265683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:37.307410 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127455) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 30 09:56:37.665596 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6ff651c8-4433-4e95-88a9-de57fc5c9f1d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "POST /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/action" status: 202 len: 0 microversion: 2.20 time: 0.413625 Sep 30 09:56:37.666358 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 483/976] 10.222.0.39 () {62 vars in 1319 bytes} [Fri Sep 30 09:56:37 2022] POST /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/action => generated 0 bytes in 415 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:56:37.674647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:37.676360 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:37.683792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:37.684293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:37.684731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:37.685444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:37.685906 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:37.686331 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:37.818049 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:37.828311 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:37.829758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:37.830938 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0de4ea56-95bd-46b3-a6da-0f9009bc5869 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1672 microversion: 2.20 time: 0.160373 Sep 30 09:56:37.831499 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 494/977] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:37 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1672 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:38.857975 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:38.858528 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:38.864501 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:38.864989 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:38.865411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:38.866093 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:38.866535 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:38.866976 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:39.001094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:39.009273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:39.010755 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:39.011895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e5bbb802-416a-4826-a20f-089aaeda7605 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1672 microversion: 2.20 time: 0.158490 Sep 30 09:56:39.012447 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 484/978] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:38 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1672 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:40.044101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:40.044692 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:40.051193 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:40.051744 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:40.052186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:40.052888 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:40.053344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:40.053771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:40.183179 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:40.191223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:40.192662 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:40.193830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bf75d8b9-0e6b-4036-a577-adf54bd8284f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.154199 Sep 30 09:56:40.194577 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 495/979] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:40 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:41.220831 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:41.221445 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:41.228522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:41.228990 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:41.229416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:41.230386 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:41.230871 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:41.231290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:41.375277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:41.384564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:41.386193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:41.387428 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d90fb175-da96-4703-8e1d-d7d7e68e972f tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.171361 Sep 30 09:56:41.396652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 485/980] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:41 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:42.413553 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:42.414108 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:42.420866 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:42.421382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:42.421813 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:42.422670 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:42.423133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:42.423560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:42.569188 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:42.573146 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:42.574611 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:42.575794 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f55aca60-e50b-4407-b1b6-30dfd0a6ff6d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.166312 Sep 30 09:56:42.576337 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 496/981] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:42 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:43.602708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:43.603298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:43.609914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:43.610400 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:43.610902 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:43.611597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:43.612053 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:43.612479 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:43.761207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:43.769630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:43.771124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:43.772266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a49b9325-8a09-4077-8b20-63b3eaff66c1 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.174147 Sep 30 09:56:43.773135 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 486/982] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:43 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:44.798489 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:44.799163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:44.805873 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:44.806525 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:44.807053 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:44.807928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:44.808532 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:44.809021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:44.949288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:44.957792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:44.959426 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:44.960658 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-456ad3cc-6584-433c-9114-e2ba036c1232 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.166101 Sep 30 09:56:44.961268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 497/983] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:44 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:45.989625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:45.990295 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:45.997013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:45.997567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:45.997996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:45.998860 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:45.999307 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:45.999719 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:46.138345 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:46.146738 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:46.148192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:46.149349 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-167342bf-d4bd-4e6d-b195-2ebda4a6eaf3 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.163812 Sep 30 09:56:46.150113 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 487/984] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:45 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:47.180674 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:47.181422 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:47.188473 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:47.189036 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:47.189522 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:47.190260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:47.190758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:47.191251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:47.331186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:47.340569 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:47.342165 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:47.343401 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-13f721c9-e111-4c3b-9adf-5bece9f62c62 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.172127 Sep 30 09:56:47.355089 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 498/985] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:47 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:48.370555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:48.371305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:48.377671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:48.378156 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:48.378605 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:48.379315 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:48.379766 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:48.380187 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:48.515150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:48.524255 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:48.525691 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:48.526866 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-937cb956-ba28-484c-b334-7caa906b61d8 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.160230 Sep 30 09:56:48.527412 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 488/986] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:48 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:49.552620 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:49.553249 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:49.559679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:49.560152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:49.560577 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:49.561261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:49.561710 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:49.562128 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:49.923229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:49.932498 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:49.933930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:49.935136 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-acce196a-d101-47d0-b2fd-d96bf25edbd0 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.386494 Sep 30 09:56:49.935705 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 499/987] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:49 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:50.961033 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:50.961577 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:50.968377 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:50.968851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:50.969279 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:50.970122 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:50.970577 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:50.971018 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:51.096905 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:51.104870 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:51.106350 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:51.107527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-807a9939-7d5c-41bc-840f-3f3e447bf2e2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.150684 Sep 30 09:56:51.108088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 489/988] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:50 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:52.133526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:52.134131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:52.140498 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:52.140958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:52.141386 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:52.142067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:52.142515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:52.142958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:52.272382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:52.280341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:52.285138 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:52.285138 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-3029795e-8d9f-4d6a-b944-d9c33e2968f4 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.153551 Sep 30 09:56:52.285138 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 500/989] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:52 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:53.309115 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:53.309716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:53.316081 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:53.321858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:53.321858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:53.323769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:53.323769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:53.323769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:53.493769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:53.493769 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:53.494014 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:53.494014 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-841d6b4e-537e-492d-a591-1d5c8183158b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.170755 Sep 30 09:56:53.494014 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 490/990] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:53 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:54.253185 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-26625cfb-eeef-4d30-8a23-7020ba217233 req-ee610a58-b83e-4810-8ccb-5b324de8b0a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "299c80c2-f890-43f2-9e42-86f4939c18df", "name": "network-vif-plugged", "status": "completed", "tag": "5fe64bf6-a448-427d-8a92-4b647a91c107"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:56:54.263921 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-26625cfb-eeef-4d30-8a23-7020ba217233 req-ee610a58-b83e-4810-8ccb-5b324de8b0a2 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:54.264401 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-26625cfb-eeef-4d30-8a23-7020ba217233 req-ee610a58-b83e-4810-8ccb-5b324de8b0a2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:54.264831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-26625cfb-eeef-4d30-8a23-7020ba217233 req-ee610a58-b83e-4810-8ccb-5b324de8b0a2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:54.276018 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-26625cfb-eeef-4d30-8a23-7020ba217233 req-ee610a58-b83e-4810-8ccb-5b324de8b0a2 service nova] Creating event network-vif-plugged:5fe64bf6-a448-427d-8a92-4b647a91c107 for instance 299c80c2-f890-43f2-9e42-86f4939c18df on hv-cinder-maste Sep 30 09:56:54.281043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-26625cfb-eeef-4d30-8a23-7020ba217233 req-ee610a58-b83e-4810-8ccb-5b324de8b0a2 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031684 Sep 30 09:56:54.281601 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 501/991] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:56:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:56:54.501405 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:54.502095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:54.508900 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:54.509400 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:54.510308 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:54.511031 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:54.511486 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:54.511908 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:54.647716 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:54.656597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:54.658181 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:54.659591 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-99e8f7e4-141a-4252-b438-eaff09a23c7e tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.162171 Sep 30 09:56:54.660219 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 491/992] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:54 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:54.739890 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-2078392097", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1635713456", "security_groups": [{"name": "tempest-secgroup-smoke-844905994"}], "block_device_mapping_v2": [{"uuid": "fcd3f7f1-28e6-4027-ad0a-ce3bff169dc9", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3408b8b1-1a84-4fa0-8af6-da7419c2525d"}]}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:56:54.852442 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] validate_networks() for [('3408b8b1-1a84-4fa0-8af6-da7419c2525d', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:56:55.226582 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:56:55.227022 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:56:55.227487 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:56:55.227926 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:56:55.228822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:56:55.237470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:55.237946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:55.238373 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:55.239144 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:55.239637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:55.240038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:55.257100 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Getting quotas for project f0d0c96bbb304882b1f2459361551a93. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:56:55.261320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Getting quotas for user c1df4fe7054840feac0a648ce82eea1a and project f0d0c96bbb304882b1f2459361551a93. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:56:55.315657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:56:55.317861 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='fcd3f7f1-28e6-4027-ad0a-ce3bff169dc9',volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:56:55.676937 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-708f0ce1-43fc-42bd-ac32-876374f4e55b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.941162 Sep 30 09:56:55.677585 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 502/993] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:56:54 2022] POST /compute/v2.1/servers => generated 405 bytes in 943 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:56:55.684926 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:55.685528 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:55.688254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-72c803d3-2ade-4d23-b073-41a3b73bd766 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:55.688865 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-72c803d3-2ade-4d23-b073-41a3b73bd766 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:55.694470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:55.695000 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:55.695433 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:55.696470 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:55.696593 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:55.696978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:55.745970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-72c803d3-2ade-4d23-b073-41a3b73bd766 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:55.753795 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-72c803d3-2ade-4d23-b073-41a3b73bd766 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1095 microversion: 2.1 time: 0.069127 Sep 30 09:56:55.754375 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 503/994] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:56:55 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1095 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:56:55.846455 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:55.854652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:55.856088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:55.857280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-cb3e323a-de81-46b2-b4d6-4865c0936d10 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.176488 Sep 30 09:56:55.857825 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 492/995] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:55 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 30 09:56:56.776067 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:56.776707 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:56.784309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:56.784792 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:56.785252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:56.790355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:56.790355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:56.790355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:56.851951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:56.861754 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0e4ef7ec-0053-42ff-a753-88db5697b245 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1145 microversion: 2.1 time: 0.089692 Sep 30 09:56:56.874647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 504/996] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:56:56 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1145 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:56:56.882537 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:56.883197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:56.889747 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:56.890225 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:56.890797 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:56.891373 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:56.891819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:56.892236 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:57.021732 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:57.030416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:57.031910 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:57.033059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-04a42bcb-27e0-4f2f-b33e-fff32bd17341 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1730 microversion: 2.20 time: 0.154849 Sep 30 09:56:57.033623 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 493/997] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:56 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1730 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:57.885716 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:57.886328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:57.892936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:57.893420 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:57.893863 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:57.894556 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:57.895038 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:57.895597 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:57.958341 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:57.966515 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1cf08f71-d0d8-42dd-80c0-633c734c5077 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1193 microversion: 2.1 time: 0.085593 Sep 30 09:56:57.967446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 505/998] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:56:57 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1193 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:56:58.058894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:58.059508 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:58.066169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:58.066729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:58.067221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:58.067916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:58.068370 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:58.068831 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:58.073380 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:56:58.219098 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:58.227536 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:58.228986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:58.230175 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-79536870-c25d-46db-aa54-bcc9a88df957 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1717 microversion: 2.20 time: 0.175664 Sep 30 09:56:58.230843 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 494/999] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:56:58 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1717 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:56:58.991000 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:56:58.991729 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:56:58.998743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:58.999314 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:59.000084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:59.001021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:59.001894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:59.001992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:59.145021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:56:59.153540 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-79aba8ce-66a5-4460-8387-4f9329c35cf7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1278 microversion: 2.1 time: 0.167794 Sep 30 09:56:59.154117 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 506/1000] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:56:58 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1278 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:56:59.804638 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-8499947f-2635-4f4a-b9b9-ea66a3395de2 req-03f4b398-2774-4603-8968-3c4259361916 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fe3f6880-acc5-4b79-9e64-e6819767007d", "tag": "31deec85-f41f-469f-8cb8-d5828a5ff711"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:56:59.809544 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=36,nova_api:UPDATE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:56:59.814973 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-8499947f-2635-4f4a-b9b9-ea66a3395de2 req-03f4b398-2774-4603-8968-3c4259361916 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:56:59.815931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-8499947f-2635-4f4a-b9b9-ea66a3395de2 req-03f4b398-2774-4603-8968-3c4259361916 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:56:59.815931 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-8499947f-2635-4f4a-b9b9-ea66a3395de2 req-03f4b398-2774-4603-8968-3c4259361916 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:56:59.827039 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-8499947f-2635-4f4a-b9b9-ea66a3395de2 req-03f4b398-2774-4603-8968-3c4259361916 service nova] Creating event network-changed:31deec85-f41f-469f-8cb8-d5828a5ff711 for instance fe3f6880-acc5-4b79-9e64-e6819767007d on hv-cinder-maste Sep 30 09:56:59.832043 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-8499947f-2635-4f4a-b9b9-ea66a3395de2 req-03f4b398-2774-4603-8968-3c4259361916 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092911 Sep 30 09:56:59.835507 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 495/1001] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:56:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:57:00.177574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:00.178192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:00.185892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:00.186381 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:00.186849 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:00.187555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:00.188012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:00.188442 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:00.194176 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:00.323480 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:00.331290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7556fa54-dbb2-4608-aa2f-d5ffa06a1273 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.157707 Sep 30 09:57:00.331799 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 507/1002] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:00 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:01.354672 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:01.355402 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:01.361703 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:01.362182 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:01.362639 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:01.363338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:01.363792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:01.364210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:01.498701 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:01.506506 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-91b718a4-7c18-4116-a16a-2ec2f334cf6f tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.156702 Sep 30 09:57:01.507101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 496/1003] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:01 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:02.529833 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:02.530449 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:02.536901 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:02.537385 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:02.537823 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:02.538512 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:02.539005 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:02.539443 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:02.670190 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:02.678131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2590a0aa-9140-4fd4-ab64-50c5cf30fbfa tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.152213 Sep 30 09:57:02.678755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 508/1004] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:02 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:03.701298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:03.701893 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:03.708526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:03.709012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:03.709458 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:03.710148 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:03.710629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:03.711052 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:03.844190 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:03.851855 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-624c483a-d297-488b-9ccf-4049ed7cbd65 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.154457 Sep 30 09:57:03.852418 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 497/1005] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:03 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:04.875713 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:04.876273 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:04.878492 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:04.883402 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:04.883881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:04.884312 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:04.884989 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:04.885437 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:04.885856 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:05.013913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:05.021598 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7e31ff46-cfb8-4673-ab1c-b387a7a0898b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.149936 Sep 30 09:57:05.022195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 509/1006] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:04 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:06.044464 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:06.045062 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:06.051516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:06.051991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:06.052416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:06.053096 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:06.053542 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:06.053959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:06.185583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:06.193179 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1a41f2f6-1454-42df-b118-80ba0becab94 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.152546 Sep 30 09:57:06.193752 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 498/1007] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:06 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:07.217078 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:07.217724 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:07.219646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:07.220344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=8 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:07.225632 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:07.231580 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:07.231580 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:07.231580 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:07.231580 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:07.231967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:07.355845 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:07.363441 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c314f0f4-6b44-468f-99da-fbc255cf9ce6 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.150896 Sep 30 09:57:07.364000 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 510/1008] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:07 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:08.386924 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:08.387538 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:08.393997 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:08.394479 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:08.394941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:08.395635 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:08.396087 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:08.396505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:08.523978 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:08.532198 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1ccf4ad8-a485-4e58-a62d-62e22354fae0 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.149409 Sep 30 09:57:08.532787 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 499/1009] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:08 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:09.554858 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:09.555501 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:09.561871 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:09.562469 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:09.562966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:09.563694 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:09.564152 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:09.564669 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:09.698208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:09.705920 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dae27d8f-d418-407c-bc3e-a3fec94823e7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.155032 Sep 30 09:57:09.706489 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 511/1010] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:09 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:10.728215 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:10.728826 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:10.735791 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:10.736275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:10.736708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:10.737397 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:10.737852 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:10.738274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:10.869071 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:10.877275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6168d333-5375-403e-ae2b-fddaeaa98e19 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.152889 Sep 30 09:57:10.877856 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 500/1011] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:10 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:11.900265 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:11.900986 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:11.907344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:11.907834 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:11.908276 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:11.908970 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:11.909466 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:11.909894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:12.044372 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:12.052063 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c8508136-809b-4423-920c-f1ea837d014e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.155711 Sep 30 09:57:12.052633 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 512/1012] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:11 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:12.460693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-414993a6-4718-4642-858e-4d6a3c2ab423 req-71147dff-7d1c-4e4b-aa77-e2ee7cd5cd0a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fe3f6880-acc5-4b79-9e64-e6819767007d", "name": "network-vif-plugged", "status": "completed", "tag": "31deec85-f41f-469f-8cb8-d5828a5ff711"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:57:12.470704 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-414993a6-4718-4642-858e-4d6a3c2ab423 req-71147dff-7d1c-4e4b-aa77-e2ee7cd5cd0a service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:12.471180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-414993a6-4718-4642-858e-4d6a3c2ab423 req-71147dff-7d1c-4e4b-aa77-e2ee7cd5cd0a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:12.471606 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-414993a6-4718-4642-858e-4d6a3c2ab423 req-71147dff-7d1c-4e4b-aa77-e2ee7cd5cd0a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:12.482320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-414993a6-4718-4642-858e-4d6a3c2ab423 req-71147dff-7d1c-4e4b-aa77-e2ee7cd5cd0a service nova] Creating event network-vif-plugged:31deec85-f41f-469f-8cb8-d5828a5ff711 for instance fe3f6880-acc5-4b79-9e64-e6819767007d on hv-cinder-maste Sep 30 09:57:12.487452 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-414993a6-4718-4642-858e-4d6a3c2ab423 req-71147dff-7d1c-4e4b-aa77-e2ee7cd5cd0a service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081840 Sep 30 09:57:12.488239 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 501/1013] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:57:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:57:13.074721 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:13.075326 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:13.081743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:13.082223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:13.082684 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:13.083399 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:13.083850 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:13.084271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:13.212485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:13.220175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-59e27ccd-5e3b-4b74-8a3e-d396966aca03 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1266 microversion: 2.1 time: 0.149399 Sep 30 09:57:13.220736 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 513/1014] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:13 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:14.244006 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:14.245265 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:14.251999 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:14.252480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:14.252913 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:14.253604 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:14.254059 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:14.254500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:14.384477 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:14.392881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-16f105b0-af04-4283-b271-dd4c2d9e39a9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1453 microversion: 2.1 time: 0.153152 Sep 30 09:57:14.393448 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 502/1015] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:14 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1453 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:14.415098 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:14.415779 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:14.421676 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:14.422158 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:14.422615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:14.423305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:14.423762 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:14.424188 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:14.555408 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:14.563782 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ccc8c7d5-0221-4b89-871a-f047d97b6981 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1453 microversion: 2.1 time: 0.152740 Sep 30 09:57:14.564342 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 514/1016] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:14 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1453 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:16.437996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-d29c8194-d737-4426-832c-7673720cfa30 req-505c7026-3787-496d-be13-67aa237e9474 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fe3f6880-acc5-4b79-9e64-e6819767007d", "tag": "31deec85-f41f-469f-8cb8-d5828a5ff711"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:57:16.447520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d29c8194-d737-4426-832c-7673720cfa30 req-505c7026-3787-496d-be13-67aa237e9474 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:16.448073 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d29c8194-d737-4426-832c-7673720cfa30 req-505c7026-3787-496d-be13-67aa237e9474 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:16.448575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-d29c8194-d737-4426-832c-7673720cfa30 req-505c7026-3787-496d-be13-67aa237e9474 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:16.459344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-d29c8194-d737-4426-832c-7673720cfa30 req-505c7026-3787-496d-be13-67aa237e9474 service nova] Creating event network-changed:31deec85-f41f-469f-8cb8-d5828a5ff711 for instance fe3f6880-acc5-4b79-9e64-e6819767007d on hv-cinder-maste Sep 30 09:57:16.464854 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-d29c8194-d737-4426-832c-7673720cfa30 req-505c7026-3787-496d-be13-67aa237e9474 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086600 Sep 30 09:57:16.465440 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 503/1017] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:57:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:57:30.514463 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:30.516727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:30.519386 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=11 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:30.520112 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=46 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:30.524899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:30.525377 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:30.525805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:30.526487 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:30.527581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:30.528039 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:30.559575 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e26bb6ff-c8b5-4e7a-bf2d-982808092ea5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.049647 Sep 30 09:57:30.562776 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 515/1018] 10.222.0.39 () {60 vars in 1343 bytes} [Fri Sep 30 09:57:30 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:30.627038 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:30.627625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:30.631745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=17 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:30.632385 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=47 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:30.635973 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:30.637266 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:30.637459 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:30.638155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:30.638623 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:30.639048 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:30.666328 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:57:30.914654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9926226d-a865-4045-a26a-0c1de61cdece tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "DELETE /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 204 len: 0 microversion: 2.20 time: 0.291490 Sep 30 09:57:30.915245 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 504/1019] 10.222.0.39 () {60 vars in 1280 bytes} [Fri Sep 30 09:57:30 2022] DELETE /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 0 bytes in 293 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 30 09:57:30.924778 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:30.926983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:30.932146 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:30.932631 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:30.933698 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:30.934395 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:30.934918 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:30.935337 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:31.064268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:31.072489 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:31.073966 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:31.075159 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5d7356a9-f0c1-44aa-8660-fc00894192c2 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.155009 Sep 30 09:57:31.075716 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 516/1020] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:57:30 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:57:32.101060 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:32.101776 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:32.108068 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:32.108547 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:32.108970 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:32.109650 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:32.110095 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:32.110509 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:32.239970 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:32.248555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:32.249979 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:32.251182 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-243ef10f-7322-49cd-ab3d-f1b34704955d tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1723 microversion: 2.20 time: 0.154433 Sep 30 09:57:32.251738 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 505/1021] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:57:32 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:57:33.277831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:33.278393 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:33.284739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:33.285212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:33.285647 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:33.286350 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:33.286827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:33.287253 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:33.349123 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:33.357369 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:33.358942 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:33.360087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-91952eb4-853a-4576-a5c4-d834733148f5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1656 microversion: 2.20 time: 0.086907 Sep 30 09:57:33.360840 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 517/1022] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:57:33 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1656 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:57:33.487494 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-0f36e9ec-8094-4e74-b0ef-c01293353f5d req-351ff0b1-0ca4-435e-82b8-4b20e26fb4fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "299c80c2-f890-43f2-9e42-86f4939c18df", "name": "network-vif-deleted", "tag": "5fe64bf6-a448-427d-8a92-4b647a91c107"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:57:33.493702 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0f36e9ec-8094-4e74-b0ef-c01293353f5d req-351ff0b1-0ca4-435e-82b8-4b20e26fb4fd service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:33.494192 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0f36e9ec-8094-4e74-b0ef-c01293353f5d req-351ff0b1-0ca4-435e-82b8-4b20e26fb4fd service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:33.494630 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-0f36e9ec-8094-4e74-b0ef-c01293353f5d req-351ff0b1-0ca4-435e-82b8-4b20e26fb4fd service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:33.505693 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-0f36e9ec-8094-4e74-b0ef-c01293353f5d req-351ff0b1-0ca4-435e-82b8-4b20e26fb4fd service nova] Creating event network-vif-deleted:5fe64bf6-a448-427d-8a92-4b647a91c107 for instance 299c80c2-f890-43f2-9e42-86f4939c18df on hv-cinder-maste Sep 30 09:57:33.510764 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-0f36e9ec-8094-4e74-b0ef-c01293353f5d req-351ff0b1-0ca4-435e-82b8-4b20e26fb4fd service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030859 Sep 30 09:57:33.516221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 506/1023] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:57:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:57:34.386169 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:34.386904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:34.393658 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:34.395992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:34.395992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:34.395992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:34.395992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:34.396499 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:34.428285 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:34.428771 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:34.429196 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:34.429641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lazy-loading 'fault' on Instance uuid 299c80c2-f890-43f2-9e42-86f4939c18df {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:57:34.473999 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:34.485480 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:34.487939 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '21d5e60badb846fb840a65f443728a9b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1142014cc9284cf1833e5bfbcdb41599', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:34.488640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-260c6f75-800d-43ab-ac8b-fc470327631b tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 200 len: 1372 microversion: 2.20 time: 0.106903 Sep 30 09:57:34.489440 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 518/1024] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:57:34 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 1372 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 30 09:57:35.515819 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:35.516526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:35.522646 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:35.523121 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:35.523599 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:35.524282 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:35.524729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:35.525146 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:35.538288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] HTTP exception thrown: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Request method failure captured: Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df HTTP/1.1 Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrw-SBto1FeGFMxbdtMei32cZ7--DPKSAV2_AfLxBMLTGswXrtFtfcO2kIRYC2gSxbY9nedY0N1c3gCVrhIn0pqN4B2bnce3vgn6MluzHM9gAFz2QnvvHxybzWs0PFbKAPJ43ncFan3dI767urivXMaYwI_5gLRsRBe2eFIgKoQtVZ0 Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Openstack-Nova-Api-Version: 2.20 Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:57:35.542848 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 21d5e60badb846fb840a65f443728a9b Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:35.544688 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '299c80c2-f890-43f2-9e42-86f4939c18df'} Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:57:35.545671 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:35.547532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:35.549003 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Returning 404 to user: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:57:35.549003 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f7304f92-af99-431d-9687-1d8bb85bce3a tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 404 len: 111 microversion: 2.20 time: 0.030367 Sep 30 09:57:35.549003 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 507/1025] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:57:35 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 30 09:57:35.552781 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4b816329-e2d9-4a13-8fce-9c9b2d6ced73 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '.version_select at 0x7f72c099e0d0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:35.557260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR oslo.messaging._drivers.impl_rabbit [None req-4b816329-e2d9-4a13-8fce-9c9b2d6ced73 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [1936d0e6-5143-4334-af9a-793f5aa6a285] AMQP server on 10.222.0.39:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 30 09:57:36.564818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO oslo.messaging._drivers.impl_rabbit [None req-4b816329-e2d9-4a13-8fce-9c9b2d6ced73 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [1936d0e6-5143-4334-af9a-793f5aa6a285] Reconnected to AMQP server on 10.222.0.39:5672 via [amqp] client with port 52212. Sep 30 09:57:36.641508 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4b816329-e2d9-4a13-8fce-9c9b2d6ced73 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1715098926" status: 204 len: 0 microversion: 2.20 time: 1.093072 Sep 30 09:57:36.642101 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 519/1026] 10.222.0.39 () {60 vars in 1262 bytes} [Fri Sep 30 09:57:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1715098926 => generated 0 bytes in 1095 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 30 09:57:38.596694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:38.597197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:38.603649 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:38.604127 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:38.605050 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:38.605729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:38.606177 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:38.606614 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:38.619780 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] HTTP exception thrown: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Request method failure captured: Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: DELETE /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df HTTP/1.1 Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrw-SBto1FeGFMxbdtMei32cZ7--DPKSAV2_AfLxBMLTGswXrtFtfcO2kIRYC2gSxbY9nedY0N1c3gCVrhIn0pqN4B2bnce3vgn6MluzHM9gAFz2QnvvHxybzWs0PFbKAPJ43ncFan3dI767urivXMaYwI_5gLRsRBe2eFIgKoQtVZ0 Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Openstack-Nova-Api-Version: 2.20 Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:57:38.621949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 21d5e60badb846fb840a65f443728a9b Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.623840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '299c80c2-f890-43f2-9e42-86f4939c18df'} Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:57:38.625015 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.626749 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.628387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Returning 404 to user: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:57:38.628387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c6c25f4d-2eb1-44e7-8a08-444d2a93543c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "DELETE /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 404 len: 111 microversion: 2.20 time: 0.030691 Sep 30 09:57:38.628387 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 508/1027] 10.222.0.39 () {60 vars in 1280 bytes} [Fri Sep 30 09:57:38 2022] DELETE /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 30 09:57:38.633637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:38.634281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: 299c80c2-f890-43f2-9e42-86f4939c18df] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:38.640143 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:38.640655 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:38.641383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:38.641773 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:38.642227 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:38.642677 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:38.655598 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] HTTP exception thrown: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Request method failure captured: Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df HTTP/1.1 Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrw-SBto1FeGFMxbdtMei32cZ7--DPKSAV2_AfLxBMLTGswXrtFtfcO2kIRYC2gSxbY9nedY0N1c3gCVrhIn0pqN4B2bnce3vgn6MluzHM9gAFz2QnvvHxybzWs0PFbKAPJ43ncFan3dI767urivXMaYwI_5gLRsRBe2eFIgKoQtVZ0 Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Openstack-Nova-Api-Version: 2.20 Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:57:38.657271 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 21d5e60badb846fb840a65f443728a9b Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.659043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '299c80c2-f890-43f2-9e42-86f4939c18df'} Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:57:38.660096 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. Sep 30 09:57:38.661743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:38.663659 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Returning 404 to user: Instance 299c80c2-f890-43f2-9e42-86f4939c18df could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:57:38.663659 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-38f41a44-e58d-4c55-b79e-ac9db384bda5 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df" status: 404 len: 111 microversion: 2.20 time: 0.029914 Sep 30 09:57:38.663659 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 520/1028] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:57:38 2022] GET /compute/v2.1/servers/299c80c2-f890-43f2-9e42-86f4939c18df => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 30 09:57:39.926943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:39.927432 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:39.934286 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:39.934809 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:39.935244 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:39.935932 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:39.936382 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:39.936800 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:39.949972 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] HTTP exception thrown: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Request method failure captured: Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: DELETE /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 HTTP/1.1 Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrw-SBto1FeGFMxbdtMei32cZ7--DPKSAV2_AfLxBMLTGswXrtFtfcO2kIRYC2gSxbY9nedY0N1c3gCVrhIn0pqN4B2bnce3vgn6MluzHM9gAFz2QnvvHxybzWs0PFbKAPJ43ncFan3dI767urivXMaYwI_5gLRsRBe2eFIgKoQtVZ0 Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Openstack-Nova-Api-Version: 2.20 Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:57:39.951526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 21d5e60badb846fb840a65f443728a9b Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.953320 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': 'dbe90c99-924b-4f03-85d5-4bfa0a6cd051'} Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:57:39.954193 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.955592 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.956920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Returning 404 to user: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:57:39.956920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3ecd3c4b-14f7-4d6d-93da-2fd4531bb54c tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "DELETE /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 404 len: 111 microversion: 2.20 time: 0.029892 Sep 30 09:57:39.956920 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 509/1029] 10.222.0.39 () {60 vars in 1280 bytes} [Fri Sep 30 09:57:39 2022] DELETE /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 30 09:57:39.961470 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:39.962087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] [instance: dbe90c99-924b-4f03-85d5-4bfa0a6cd051] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:39.968658 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:39.969150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:39.969591 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:39.970287 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:39.970787 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:39.971221 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:39.984764 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] HTTP exception thrown: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Request method failure captured: Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 HTTP/1.1 Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrw-SBto1FeGFMxbdtMei32cZ7--DPKSAV2_AfLxBMLTGswXrtFtfcO2kIRYC2gSxbY9nedY0N1c3gCVrhIn0pqN4B2bnce3vgn6MluzHM9gAFz2QnvvHxybzWs0PFbKAPJ43ncFan3dI767urivXMaYwI_5gLRsRBe2eFIgKoQtVZ0 Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Openstack-Nova-Api-Version: 2.20 Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:57:39.986286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_1142014cc9284cf1833e5bfbcdb41599"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 1142014cc9284cf1833e5bfbcdb41599 Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1331107251 Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 21d5e60badb846fb840a65f443728a9b Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1331107251-project Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.988021 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': 'dbe90c99-924b-4f03-85d5-4bfa0a6cd051'} Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:57:39.989155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. Sep 30 09:57:39.990712 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:57:39.992186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] Returning 404 to user: Instance dbe90c99-924b-4f03-85d5-4bfa0a6cd051 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:57:39.992186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c2b9021a-9943-45b1-979d-8a74263ff2f7 tempest-AttachVolumeShelveTestJSON-1331107251 tempest-AttachVolumeShelveTestJSON-1331107251-project] 10.222.0.39 "GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051" status: 404 len: 111 microversion: 2.20 time: 0.030150 Sep 30 09:57:39.992186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 521/1030] 10.222.0.39 () {60 vars in 1277 bytes} [Fri Sep 30 09:57:39 2022] GET /compute/v2.1/servers/dbe90c99-924b-4f03-85d5-4bfa0a6cd051 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 30 09:57:50.510758 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=17,nova_api:UPDATE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:50.515925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:50.629367 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-218d86e4-756c-4497-bc55-49895796e7c1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'create', calling method: .version_select at 0x7f72c0aac4c0>, body: {"keypair": {"name": "tempest-keypair-1308355648", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE+Dfg28SEBegaJjkuux7Oun06tZ7zYqloRZzpr1UB3pF/DJ/NnngOuXJDIykMsnmt7wtQ1iWZp7keINeBmRCErDdhAWk8b1ukIzmqULAzT64+S0MT6LvmS4V1e5JYZu5A=="}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:57:50.648049 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-218d86e4-756c-4497-bc55-49895796e7c1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Getting quotas for project 85eef8fd790146cfa7da6214c13ca262. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:57:50.653566 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-218d86e4-756c-4497-bc55-49895796e7c1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Getting quotas for user ee46fc4dd13143bfb409fb5033d9a4d2 and project 85eef8fd790146cfa7da6214c13ca262. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:57:50.784532 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-218d86e4-756c-4497-bc55-49895796e7c1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.279656 Sep 30 09:57:50.785304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 510/1031] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 09:57:50 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:57:53.048605 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-475143004", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6c765eee-bae8-4b34-897a-abb29acd5a27"}], "security_groups": [{"name": "tempest-securitygroup--868858801"}], "key_name": "tempest-keypair-1308355648", "user_data": "}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:57:53.057576 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=17,nova_api:DELETE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:53.058297 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=30 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:53.168274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] validate_networks() for [('6c765eee-bae8-4b34-897a-abb29acd5a27', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 09:57:53.605208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 09:57:53.605208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 09:57:53.605208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 09:57:53.605208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 09:57:53.605208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 09:57:53.612529 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:53.612957 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:53.613394 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:53.614204 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:53.614689 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:53.615118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:53.632903 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Getting quotas for project 85eef8fd790146cfa7da6214c13ca262. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 09:57:53.637471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Getting quotas for user ee46fc4dd13143bfb409fb5033d9a4d2 and project 85eef8fd790146cfa7da6214c13ca262. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 09:57:53.649081 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 09:57:53.651305 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 09:57:53.790982 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d1d2d79f-c04f-48ca-b595-d93d2838e73c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.746685 Sep 30 09:57:53.791713 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 522/1032] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 09:57:53 2022] POST /compute/v2.1/servers => generated 401 bytes in 748 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 09:57:53.803660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1ea7990e-f80c-436e-bec5-c0288d64a1e7 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:53.806698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1ea7990e-f80c-436e-bec5-c0288d64a1e7 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:53.870283 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1ea7990e-f80c-436e-bec5-c0288d64a1e7 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:53.878516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1ea7990e-f80c-436e-bec5-c0288d64a1e7 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1237 microversion: 2.1 time: 0.079350 Sep 30 09:57:53.879367 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 511/1033] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:53 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1237 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:54.902948 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:54.903563 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:54.909796 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:54.910286 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:54.910752 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:54.911511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:54.911988 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:54.912423 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:54.981557 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:54.989384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b8c317c5-085c-460f-8a88-4fb5ed8ca706 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1241 microversion: 2.1 time: 0.091555 Sep 30 09:57:54.989951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 523/1034] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:54 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1241 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:56.011971 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:56.012579 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:56.019680 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:56.020205 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:56.020642 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:56.021332 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:56.021787 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:56.022207 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:56.084280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:56.092684 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b4b0618d-02de-4f08-b661-dc17d708cc58 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1289 microversion: 2.1 time: 0.084631 Sep 30 09:57:56.093168 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 512/1035] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:56 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1289 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:57.117139 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:57.117740 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:57.124774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:57.125290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:57.125751 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:57.126574 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:57.127056 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:57.127486 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:57.265003 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:57.273019 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9abc7fd7-4733-4023-b561-78bd92c66ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.160008 Sep 30 09:57:57.273617 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 524/1036] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:57 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:58.046314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-b92a23c2-3dae-4080-847d-7bad44d99d8c req-97851884-113b-4c09-b093-b7cf0c2b793a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8872f2fa-9f26-40d0-b256-f8f31dd116e1", "tag": "8354001a-2686-4424-85b3-eda11d52c3b4"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:57:58.056329 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-b92a23c2-3dae-4080-847d-7bad44d99d8c req-97851884-113b-4c09-b093-b7cf0c2b793a service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:58.056802 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-b92a23c2-3dae-4080-847d-7bad44d99d8c req-97851884-113b-4c09-b093-b7cf0c2b793a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:58.057223 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-b92a23c2-3dae-4080-847d-7bad44d99d8c req-97851884-113b-4c09-b093-b7cf0c2b793a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:58.068628 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-b92a23c2-3dae-4080-847d-7bad44d99d8c req-97851884-113b-4c09-b093-b7cf0c2b793a service nova] Creating event network-changed:8354001a-2686-4424-85b3-eda11d52c3b4 for instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 on hv-cinder-maste Sep 30 09:57:58.073575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-b92a23c2-3dae-4080-847d-7bad44d99d8c req-97851884-113b-4c09-b093-b7cf0c2b793a service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032362 Sep 30 09:57:58.075048 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 513/1037] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:57:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:57:58.296557 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:58.297206 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:58.303818 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:58.304308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:58.304743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:58.305435 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:58.306012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:58.306439 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:58.438763 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:58.446847 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9466774d-f5ac-49f3-9e56-fdc8f69513f6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.154182 Sep 30 09:57:58.447412 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 525/1038] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:58 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:57:59.470489 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:57:59.471141 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:57:59.477288 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:59.477765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:59.478197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:59.478924 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:57:59.479376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:57:59.479798 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:57:59.609666 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:57:59.617275 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-32b98d63-1fd3-4ba0-acbf-313254cc09b4 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.151237 Sep 30 09:57:59.617855 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 514/1039] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:57:59 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:00.640959 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:00.641640 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:00.648366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:00.648842 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:00.649270 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:00.649952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:00.650400 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:00.650860 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:00.775830 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:00.783860 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-41aecaf7-bda2-4db7-b791-e99d8cb9013f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.146829 Sep 30 09:58:00.784423 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 526/1040] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:00 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:01.807176 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:01.807782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:01.814861 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:01.815344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:01.815778 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:01.816467 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:01.816918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:01.817343 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:01.946038 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:01.953611 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ab82d176-68d3-4b6d-9948-590992fc0384 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.150337 Sep 30 09:58:01.954181 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 515/1041] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:01 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:02.976586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:02.977190 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:02.983514 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:02.983994 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:02.984424 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:02.985103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:02.985553 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:02.985976 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:03.114366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:03.122701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1b2a0720-c8a3-41f2-b6d3-4525e66bb013 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.149941 Sep 30 09:58:03.123505 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 527/1042] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:02 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:04.146026 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:04.146667 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:04.149291 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=6 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:58:04.154678 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:04.155164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:04.155601 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:04.156289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:04.156805 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:04.157230 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:04.294169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:04.301820 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-89184b92-e98f-4c97-aeda-b310bdce2e24 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.159696 Sep 30 09:58:04.302382 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 516/1043] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:04 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:04.856206 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-cec6283d-9cb9-4c1a-86e4-c536ff208e08 req-36de9aba-c27a-4e6a-becc-44ea4c54c2c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8872f2fa-9f26-40d0-b256-f8f31dd116e1", "name": "network-vif-plugged", "status": "completed", "tag": "8354001a-2686-4424-85b3-eda11d52c3b4"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:58:04.861203 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:58:04.861877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=6 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:58:04.867570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-cec6283d-9cb9-4c1a-86e4-c536ff208e08 req-36de9aba-c27a-4e6a-becc-44ea4c54c2c6 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:04.868043 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-cec6283d-9cb9-4c1a-86e4-c536ff208e08 req-36de9aba-c27a-4e6a-becc-44ea4c54c2c6 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:04.868467 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-cec6283d-9cb9-4c1a-86e4-c536ff208e08 req-36de9aba-c27a-4e6a-becc-44ea4c54c2c6 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:04.879619 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-cec6283d-9cb9-4c1a-86e4-c536ff208e08 req-36de9aba-c27a-4e6a-becc-44ea4c54c2c6 service nova] Creating event network-vif-plugged:8354001a-2686-4424-85b3-eda11d52c3b4 for instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 on hv-cinder-maste Sep 30 09:58:04.885456 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-cec6283d-9cb9-4c1a-86e4-c536ff208e08 req-36de9aba-c27a-4e6a-becc-44ea4c54c2c6 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034020 Sep 30 09:58:04.886084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 528/1044] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:58:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:58:05.325020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:05.325881 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:05.332629 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:05.333115 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:05.333553 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:05.334258 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:05.334750 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:05.340743 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:05.462844 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:05.471017 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ab9e0101-88f6-470c-a0d8-dfe4215d4730 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.149936 Sep 30 09:58:05.471581 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 517/1045] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:05 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:06.494742 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:06.495482 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:06.503391 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:06.503877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:06.504320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:06.505019 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:06.505533 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:06.505962 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:06.641406 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:06.649106 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-68a70960-6554-4a0a-b74a-87a11a6538da tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1362 microversion: 2.1 time: 0.159260 Sep 30 09:58:06.649678 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 529/1046] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:06 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1362 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:07.671767 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:07.672365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:07.678891 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:07.679369 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:07.679797 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:07.680479 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:07.680923 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:07.681370 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:07.806925 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:07.814425 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-90324d79-1520-4fcb-8f65-f1d6a3676777 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1548 microversion: 2.1 time: 0.146509 Sep 30 09:58:07.815012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 518/1047] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:07 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1548 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:07.836827 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:07.837347 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:07.844384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:07.844863 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:07.845297 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:07.845986 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:07.846441 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:07.846902 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:07.921621 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7a89a9f5-d128-484c-ab09-efc0753c9197 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.088641 Sep 30 09:58:07.922190 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 530/1048] 10.222.0.39 () {58 vars in 1275 bytes} [Fri Sep 30 09:58:07 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/os-interface => generated 285 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:58:08.876164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-11339c7b-4d4b-4bc0-8cc9-a70107eb606d req-99b4d713-2967-49f4-a286-6629369795b2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8872f2fa-9f26-40d0-b256-f8f31dd116e1", "tag": "8354001a-2686-4424-85b3-eda11d52c3b4"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:58:08.886007 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-11339c7b-4d4b-4bc0-8cc9-a70107eb606d req-99b4d713-2967-49f4-a286-6629369795b2 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:08.886473 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-11339c7b-4d4b-4bc0-8cc9-a70107eb606d req-99b4d713-2967-49f4-a286-6629369795b2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:08.886959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-11339c7b-4d4b-4bc0-8cc9-a70107eb606d req-99b4d713-2967-49f4-a286-6629369795b2 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:08.897545 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-11339c7b-4d4b-4bc0-8cc9-a70107eb606d req-99b4d713-2967-49f4-a286-6629369795b2 service nova] Creating event network-changed:8354001a-2686-4424-85b3-eda11d52c3b4 for instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 on hv-cinder-maste Sep 30 09:58:08.904442 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-11339c7b-4d4b-4bc0-8cc9-a70107eb606d req-99b4d713-2967-49f4-a286-6629369795b2 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029848 Sep 30 09:58:08.904442 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 519/1049] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:58:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:58:44.678323 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=31,nova_api:INSERT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:58:44.679107 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=44 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:58:44.682954 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:44.683444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:44.690337 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:44.690878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:44.691307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:44.691998 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:44.692452 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:44.692909 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:44.716904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-f3ce6537-571e-414e-8284-4f17cba87616 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/ips" status: 200 len: 145 microversion: 2.1 time: 0.040239 Sep 30 09:58:44.717460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 531/1050] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 09:58:44 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/ips => generated 145 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:58:47.608733 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=31,nova_api:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:58:47.609839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=40 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:58:47.614389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ad9d918a-ca58-4a22-9731-f82ed6598509"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:58:47.616898 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:47.623273 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:47.623751 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:47.624175 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:47.624892 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:47.625364 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:47.625779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:48.334853 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1e30d9aa-9e3f-4140-8209-5a16ed912941 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.727471 Sep 30 09:58:48.335487 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 520/1051] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 09:58:47 2022] POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/os-volume_attachments => generated 194 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:58:48.736174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:48.737241 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:48.738800 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-f2f6520c-70c1-4ed9-b147-556eda1ba679 req-6eb05302-d186-4530-b7af-afa18e025822 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fe3f6880-acc5-4b79-9e64-e6819767007d", "tag": "31deec85-f41f-469f-8cb8-d5828a5ff711"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:58:48.744670 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:48.744945 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:48.745382 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:48.746155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:48.746523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:48.747017 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:48.748316 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-f2f6520c-70c1-4ed9-b147-556eda1ba679 req-6eb05302-d186-4530-b7af-afa18e025822 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:48.748789 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-f2f6520c-70c1-4ed9-b147-556eda1ba679 req-6eb05302-d186-4530-b7af-afa18e025822 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:48.749210 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-f2f6520c-70c1-4ed9-b147-556eda1ba679 req-6eb05302-d186-4530-b7af-afa18e025822 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:48.760826 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-f2f6520c-70c1-4ed9-b147-556eda1ba679 req-6eb05302-d186-4530-b7af-afa18e025822 service nova] Creating event network-changed:31deec85-f41f-469f-8cb8-d5828a5ff711 for instance fe3f6880-acc5-4b79-9e64-e6819767007d on hv-cinder-maste Sep 30 09:58:48.765738 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-f2f6520c-70c1-4ed9-b147-556eda1ba679 req-6eb05302-d186-4530-b7af-afa18e025822 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031430 Sep 30 09:58:48.766707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 521/1052] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:58:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:58:48.774006 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:58:49.039159 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-79eaa757-521b-430d-865c-e9b419d58180 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "DELETE /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 204 len: 0 microversion: 2.1 time: 0.307557 Sep 30 09:58:49.040058 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 532/1053] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:58:48 2022] DELETE /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:58:49.049196 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:49.050125 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:49.056384 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:49.056874 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:49.057335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:49.058033 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:49.058485 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:49.059008 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:49.209777 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:49.218212 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-8e53c7da-dd20-4191-bf9e-82e85e4bc573 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1578 microversion: 2.1 time: 0.173136 Sep 30 09:58:49.225794 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 522/1054] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:49 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1578 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:50.243584 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:50.244302 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:50.251517 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:50.252001 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:50.252433 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:50.253130 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:50.253587 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:50.254008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:50.394117 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:50.401886 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ab8885cf-d4d3-4185-b434-3c8b4fe58736 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1578 microversion: 2.1 time: 0.162616 Sep 30 09:58:50.402453 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 533/1055] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:50 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1578 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:51.425334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:51.425966 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:51.432391 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:51.433201 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:51.433335 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:51.433965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:51.434415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:51.434868 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:51.583396 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:51.591057 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ac488b43-c709-4c0d-948a-6a3714ec2f0b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1459 microversion: 2.1 time: 0.169800 Sep 30 09:58:51.605894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 523/1056] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:51 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1459 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:51.763022 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-de7bf80f-8f31-4699-a0e1-6ffaf1278c9c req-e367fc19-50ac-455d-8b67-3ea670a55c9a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fe3f6880-acc5-4b79-9e64-e6819767007d", "name": "network-vif-deleted", "tag": "31deec85-f41f-469f-8cb8-d5828a5ff711"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:58:51.773657 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-de7bf80f-8f31-4699-a0e1-6ffaf1278c9c req-e367fc19-50ac-455d-8b67-3ea670a55c9a service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:51.777234 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-de7bf80f-8f31-4699-a0e1-6ffaf1278c9c req-e367fc19-50ac-455d-8b67-3ea670a55c9a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:51.777234 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-de7bf80f-8f31-4699-a0e1-6ffaf1278c9c req-e367fc19-50ac-455d-8b67-3ea670a55c9a service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:51.785731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-de7bf80f-8f31-4699-a0e1-6ffaf1278c9c req-e367fc19-50ac-455d-8b67-3ea670a55c9a service nova] Creating event network-vif-deleted:31deec85-f41f-469f-8cb8-d5828a5ff711 for instance fe3f6880-acc5-4b79-9e64-e6819767007d on hv-cinder-maste Sep 30 09:58:51.791251 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-de7bf80f-8f31-4699-a0e1-6ffaf1278c9c req-e367fc19-50ac-455d-8b67-3ea670a55c9a service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032277 Sep 30 09:58:51.791928 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 534/1057] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:58:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:58:52.613000 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:52.613643 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:52.620117 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:52.620604 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:52.621037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:52.621725 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:52.622177 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:52.622617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:52.653043 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:52.653519 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:52.653993 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:52.654997 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lazy-loading 'fault' on Instance uuid fe3f6880-acc5-4b79-9e64-e6819767007d {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:58:52.708078 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:52.715878 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-243c1b7b-b49f-4a24-b7e2-494c8a11db49 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 200 len: 1231 microversion: 2.1 time: 0.106649 Sep 30 09:58:52.717085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 524/1058] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:52 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 1231 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:52.719812 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-09a42c0f-a7ed-4f28-b6ea-06a14b805cc2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:58:52.726050 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-09a42c0f-a7ed-4f28-b6ea-06a14b805cc2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:52.726604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-09a42c0f-a7ed-4f28-b6ea-06a14b805cc2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:52.727117 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-09a42c0f-a7ed-4f28-b6ea-06a14b805cc2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:52.785932 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-09a42c0f-a7ed-4f28-b6ea-06a14b805cc2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Going to try to stop instance {{(pid=127456) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 30 09:58:52.960942 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-09a42c0f-a7ed-4f28-b6ea-06a14b805cc2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action" status: 202 len: 0 microversion: 2.1 time: 0.245786 Sep 30 09:58:52.961581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 535/1059] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:58:52 2022] POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action => generated 0 bytes in 248 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 30 09:58:52.969861 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:52.970465 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:52.976376 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:52.976852 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:52.977280 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:52.977962 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:52.978474 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:52.978851 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:53.106256 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:53.114401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-54d41b7b-807e-4e06-93b3-c6e567c75594 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.148549 Sep 30 09:58:53.114997 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 525/1060] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:52 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:53.740035 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:53.740734 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: fe3f6880-acc5-4b79-9e64-e6819767007d] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:53.746877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:53.747366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:53.747801 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:53.748490 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:53.748942 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:53.749366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:53.762785 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] HTTP exception thrown: Instance fe3f6880-acc5-4b79-9e64-e6819767007d could not be found. Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Request method failure captured: Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d HTTP/1.1 Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrwbJ3m7BfeNyh8xFop0ZrSTHHKK21Y7TdEWVYOY981ZqdFX0a9H7WcQ4DWGzk0TEbHy0wWxSWu5CzdUoC4SijJ_Vsyyw0tRnkBt6GRjmAtPzOP7lkvEE6VbxsqG8jjOLgxBhtOpmz_2iXVO_1w7lEfD2STZdTrOoIC2NMHTKeuq5Sw Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:58:53.764371 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: c1df4fe7054840feac0a648ce82eea1a Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance fe3f6880-acc5-4b79-9e64-e6819767007d could not be found. Sep 30 09:58:53.766150 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': 'fe3f6880-acc5-4b79-9e64-e6819767007d'} Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fe3f6880-acc5-4b79-9e64-e6819767007d could not be found. Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:58:53.767307 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fe3f6880-acc5-4b79-9e64-e6819767007d could not be found. Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fe3f6880-acc5-4b79-9e64-e6819767007d could not be found. Sep 30 09:58:53.768952 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:58:53.770671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Returning 404 to user: Instance fe3f6880-acc5-4b79-9e64-e6819767007d could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:58:53.770671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b5b3e33f-b724-4436-9d63-6787d11a09d9 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d" status: 404 len: 111 microversion: 2.1 time: 0.030007 Sep 30 09:58:53.770671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 536/1061] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:53 2022] GET /compute/v2.1/servers/fe3f6880-acc5-4b79-9e64-e6819767007d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:58:54.138060 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:54.138696 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:54.145480 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:54.145955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:54.146384 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:54.147089 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:54.147540 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:54.147953 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:54.275390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:54.284246 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-eb01defe-ba2b-4cf2-b09a-d1cc6242ec65 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.150026 Sep 30 09:58:54.284815 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 526/1062] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:54 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:55.307279 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:55.307891 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:55.314383 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:55.314897 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:55.315335 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:55.316022 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:55.316509 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:55.316939 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:55.673426 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:55.681598 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-40f61194-1c4f-4880-b62c-c55dd1fda72d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.378144 Sep 30 09:58:55.682214 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 537/1063] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:55 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:56.704963 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:56.705503 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:56.711930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:56.712413 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:56.712845 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:56.713541 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:56.714248 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:56.714710 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:56.930915 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:56.939072 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-929bde58-1048-4f24-889b-955d48dd4d43 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.238191 Sep 30 09:58:56.939649 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 527/1064] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:56 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:57.087986 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:57.088541 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:57.090362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-635a5d74-c9b2-484a-a877-495c88c72d53 req-4d0801e7-d2e2-4c34-81fe-26aa7392c5ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3ee8e26b-68fe-4a79-a257-1d9456e4b22b", "tag": "3570f723-a6b6-479e-9662-f90c771a0449"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:58:57.095910 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:57.096653 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:57.097094 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:57.097800 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:57.098495 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:57.099020 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:57.100559 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-635a5d74-c9b2-484a-a877-495c88c72d53 req-4d0801e7-d2e2-4c34-81fe-26aa7392c5ad service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:57.101037 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-635a5d74-c9b2-484a-a877-495c88c72d53 req-4d0801e7-d2e2-4c34-81fe-26aa7392c5ad service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:57.101465 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-635a5d74-c9b2-484a-a877-495c88c72d53 req-4d0801e7-d2e2-4c34-81fe-26aa7392c5ad service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:57.112753 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-635a5d74-c9b2-484a-a877-495c88c72d53 req-4d0801e7-d2e2-4c34-81fe-26aa7392c5ad service nova] Creating event network-changed:3570f723-a6b6-479e-9662-f90c771a0449 for instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b on hv-cinder-maste Sep 30 09:58:57.119314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-635a5d74-c9b2-484a-a877-495c88c72d53 req-4d0801e7-d2e2-4c34-81fe-26aa7392c5ad service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033310 Sep 30 09:58:57.120162 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 528/1065] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:58:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 30 09:58:57.125765 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 09:58:57.393208 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b7c5fbf8-795e-42ed-a6ad-a23e183d3ea7 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "DELETE /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 204 len: 0 microversion: 2.1 time: 0.309317 Sep 30 09:58:57.393797 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 538/1066] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:58:57 2022] DELETE /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 09:58:57.402647 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:57.403365 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:57.409654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:57.410141 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:57.410581 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:57.411303 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:57.411760 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:57.412182 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:57.549842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:57.557682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0a748613-528d-41d8-9650-20e193f7c467 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1577 microversion: 2.1 time: 0.159358 Sep 30 09:58:57.558245 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 529/1067] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:57 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1577 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:57.963033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:57.963664 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:57.969561 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:57.970044 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:57.970481 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:57.971212 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:57.971671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:57.972097 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:58.100147 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:58.108260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b659ceed-f099-4f26-9a38-5998570e89d2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.149564 Sep 30 09:58:58.108824 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 539/1068] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:57 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:58.581785 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:58.582394 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:58.588923 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:58.589403 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:58.589839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:58.590525 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:58.591031 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:58.591453 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:58.728014 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:58.735730 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-05a3aafc-c7af-4c21-b2ce-689363649d34 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1577 microversion: 2.1 time: 0.158346 Sep 30 09:58:58.736297 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 530/1069] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:58 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1577 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:59.133618 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:59.134330 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:59.141361 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:59.141843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:59.142274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:59.142983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:59.143434 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:59.143851 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:59.270795 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:59.279349 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-fee88fc3-7029-476a-ac77-57efb36a9079 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.149881 Sep 30 09:58:59.279914 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 540/1070] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:59 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:58:59.759298 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:58:59.759910 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:58:59.766726 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:59.767220 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:59.767644 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:59.768585 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:59.769044 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:59.769179 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-efc03d05-d46a-414e-a6b8-8301aeee393c req-4fd3f386-bd41-48dd-bdd6-1fd428c9008d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3ee8e26b-68fe-4a79-a257-1d9456e4b22b", "name": "network-vif-deleted", "tag": "3570f723-a6b6-479e-9662-f90c771a0449"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:58:59.769464 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:59.779154 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-efc03d05-d46a-414e-a6b8-8301aeee393c req-4fd3f386-bd41-48dd-bdd6-1fd428c9008d service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:58:59.779951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-efc03d05-d46a-414e-a6b8-8301aeee393c req-4fd3f386-bd41-48dd-bdd6-1fd428c9008d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:58:59.780381 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-efc03d05-d46a-414e-a6b8-8301aeee393c req-4fd3f386-bd41-48dd-bdd6-1fd428c9008d service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:58:59.792111 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-efc03d05-d46a-414e-a6b8-8301aeee393c req-4fd3f386-bd41-48dd-bdd6-1fd428c9008d service nova] Creating event network-vif-deleted:3570f723-a6b6-479e-9662-f90c771a0449 for instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b on hv-cinder-maste Sep 30 09:58:59.797972 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-efc03d05-d46a-414e-a6b8-8301aeee393c req-4fd3f386-bd41-48dd-bdd6-1fd428c9008d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032502 Sep 30 09:58:59.798569 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 541/1071] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 09:58:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 09:58:59.832012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:58:59.840132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2ccb7c70-1c4a-4474-bafc-b094d5b864bc tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1392 microversion: 2.1 time: 0.085031 Sep 30 09:58:59.841122 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 531/1072] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:58:59 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1392 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:00.303336 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:00.303971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:00.310707 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:00.311221 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:00.311645 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:00.312344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:00.312805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:00.313236 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:00.452396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:00.459154 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-071a1ece-8e5e-42a7-a62e-c59f74296ffe tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.160136 Sep 30 09:59:00.459728 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 542/1073] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:00 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:00.864221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:00.864894 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:00.871276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:00.871761 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:00.872196 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:00.872895 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:00.873352 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:00.873775 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:00.904782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:00.905260 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:00.905699 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:00.906169 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lazy-loading 'fault' on Instance uuid 3ee8e26b-68fe-4a79-a257-1d9456e4b22b {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:59:00.947108 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1df4fe7054840feac0a648ce82eea1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0d0c96bbb304882b1f2459361551a93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:00.954754 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-40cf1e69-1c91-4641-a4d8-8c287a594c4b tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 200 len: 1231 microversion: 2.1 time: 0.095150 Sep 30 09:59:00.955323 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 532/1074] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:00 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 1231 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:01.482551 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:01.483184 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:01.489499 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:01.489971 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:01.490396 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:01.491111 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:01.491562 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:01.492015 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:01.620694 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:01.628785 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d26955df-3df7-4da1-b78f-a0052176cc31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.150113 Sep 30 09:59:01.629345 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 543/1075] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:01 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:01.977580 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:01.978189 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: 3ee8e26b-68fe-4a79-a257-1d9456e4b22b] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:01.984656 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:01.985143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:01.985578 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:01.986274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:01.986758 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:01.987186 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:02.000700 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] HTTP exception thrown: Instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b could not be found. Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Request method failure captured: Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b HTTP/1.1 Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrwbJ3m7BfeNyh8xFop0ZrSTHHKK21Y7TdEWVYOY981ZqdFX0a9H7WcQ4DWGzk0TEbHy0wWxSWu5CzdUoC4SijJ_Vsyyw0tRnkBt6GRjmAtPzOP7lkvEE6VbxsqG8jjOLgxBhtOpmz_2iXVO_1w7lEfD2STZdTrOoIC2NMHTKeuq5Sw Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:59:02.002311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: c1df4fe7054840feac0a648ce82eea1a Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b could not be found. Sep 30 09:59:02.004075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '3ee8e26b-68fe-4a79-a257-1d9456e4b22b'} Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b could not be found. Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:59:02.005213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b could not be found. Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b could not be found. Sep 30 09:59:02.007099 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.008729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Returning 404 to user: Instance 3ee8e26b-68fe-4a79-a257-1d9456e4b22b could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:59:02.008729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-881de8f1-3103-4943-8645-c5b2ba93289c tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b" status: 404 len: 111 microversion: 2.1 time: 0.030064 Sep 30 09:59:02.008729 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 533/1076] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:01 2022] GET /compute/v2.1/servers/3ee8e26b-68fe-4a79-a257-1d9456e4b22b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:59:02.544463 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:02.544945 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:02.551793 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:02.552272 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:02.552703 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:02.553388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:02.553837 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:02.554254 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:02.567904 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] HTTP exception thrown: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Request method failure captured: Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 HTTP/1.1 Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNrwbJ3m7BfeNyh8xFop0ZrSTHHKK21Y7TdEWVYOY981ZqdFX0a9H7WcQ4DWGzk0TEbHy0wWxSWu5CzdUoC4SijJ_Vsyyw0tRnkBt6GRjmAtPzOP7lkvEE6VbxsqG8jjOLgxBhtOpmz_2iXVO_1w7lEfD2STZdTrOoIC2NMHTKeuq5Sw Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 09:59:02.569544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: c1df4fe7054840feac0a648ce82eea1a Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.571268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': 'bcd7505a-8f4d-4b0b-969e-776089b2e6d1'} Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:59:02.572388 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.573960 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.575755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Returning 404 to user: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:59:02.575755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-74e8e9d8-4965-4fac-9a47-6e600d56854e tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "DELETE /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 404 len: 111 microversion: 2.1 time: 0.030329 Sep 30 09:59:02.575755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 544/1077] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 09:59:02 2022] DELETE /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:59:02.579238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:02.579846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] [instance: bcd7505a-8f4d-4b0b-969e-776089b2e6d1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:02.585705 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:02.586182 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:02.586631 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:02.587317 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:02.587767 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:02.588191 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:02.601771 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] HTTP exception thrown: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Request method failure captured: Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 HTTP/1.1 Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNrwbJ3m7BfeNyh8xFop0ZrSTHHKK21Y7TdEWVYOY981ZqdFX0a9H7WcQ4DWGzk0TEbHy0wWxSWu5CzdUoC4SijJ_Vsyyw0tRnkBt6GRjmAtPzOP7lkvEE6VbxsqG8jjOLgxBhtOpmz_2iXVO_1w7lEfD2STZdTrOoIC2NMHTKeuq5Sw Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 09:59:02.603290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_f0d0c96bbb304882b1f2459361551a93"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: f0d0c96bbb304882b1f2459361551a93 Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-TestVolumeBootPattern-1826206613 Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: c1df4fe7054840feac0a648ce82eea1a Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-TestVolumeBootPattern-1826206613-project Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.605226 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': 'bcd7505a-8f4d-4b0b-969e-776089b2e6d1'} Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 09:59:02.606354 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. Sep 30 09:59:02.608010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 09:59:02.609984 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Returning 404 to user: Instance bcd7505a-8f4d-4b0b-969e-776089b2e6d1 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 09:59:02.609984 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4e17e5a8-392c-431d-a95d-f08b8a7960f4 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1" status: 404 len: 111 microversion: 2.1 time: 0.029152 Sep 30 09:59:02.609984 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 534/1078] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:02 2022] GET /compute/v2.1/servers/bcd7505a-8f4d-4b0b-969e-776089b2e6d1 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 09:59:02.652511 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:02.653149 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:02.659900 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:02.660386 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:02.660826 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:02.661579 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:02.662037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:02.662464 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:02.795739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:02.804755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-495d0443-5f62-462a-834e-091f7664dcfb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1722 microversion: 2.1 time: 0.156116 Sep 30 09:59:02.805905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 545/1079] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:02 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:03.827793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:03.828344 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:03.835021 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:03.835512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:03.835948 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:03.836648 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:03.837105 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:03.837530 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:04.043711 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:04.052669 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a7de1e39-d745-41c0-96b3-ac4d77559057 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1699 microversion: 2.1 time: 0.228894 Sep 30 09:59:04.053243 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 535/1080] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:03 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1699 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:59:04.075779 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9d2827ba-b425-49c6-83e9-17cbf0bac4ff tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:59:04.082230 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d2827ba-b425-49c6-83e9-17cbf0bac4ff tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:04.082680 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d2827ba-b425-49c6-83e9-17cbf0bac4ff tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:04.083279 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9d2827ba-b425-49c6-83e9-17cbf0bac4ff tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:04.086822 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:59:04.105137 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9d2827ba-b425-49c6-83e9-17cbf0bac4ff tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Going to try to start instance {{(pid=127456) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 30 09:59:04.157520 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-3d58a371-bf1a-4ae3-94c0-57cfe615bf26 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] Calling method '.version_select at 0x7f72c0bdbca0>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:04.262713 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-3d58a371-bf1a-4ae3-94c0-57cfe615bf26 tempest-TestVolumeBootPattern-1826206613 tempest-TestVolumeBootPattern-1826206613-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1635713456" status: 202 len: 0 microversion: 2.1 time: 0.109027 Sep 30 09:59:04.263490 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 536/1081] 10.222.0.39 () {58 vars in 1263 bytes} [Fri Sep 30 09:59:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1635713456 => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 30 09:59:04.355878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9d2827ba-b425-49c6-83e9-17cbf0bac4ff tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action" status: 202 len: 0 microversion: 2.1 time: 0.283837 Sep 30 09:59:04.356578 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 546/1082] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:59:04 2022] POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action => generated 0 bytes in 285 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 30 09:59:04.365324 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:04.365918 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:04.428637 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:04.429274 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:04.429793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:04.430539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:04.431195 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:04.431617 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:04.651311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:04.659586 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-006a9632-36f0-4ec7-ac40-1a12a2ab3e2c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1708 microversion: 2.1 time: 0.298737 Sep 30 09:59:04.660168 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 537/1083] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:04 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1708 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:05.684839 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:05.685460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:05.693090 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:05.693581 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:05.694018 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:05.694779 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:05.695238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:05.695665 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:06.030544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:06.038973 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-62e5bd46-5f9b-4390-bf85-88e6c984b963 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1708 microversion: 2.1 time: 0.359170 Sep 30 09:59:06.039555 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 547/1084] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:05 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1708 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:07.062318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:07.063054 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:07.069523 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:07.070001 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:07.070426 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:07.071149 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:07.071595 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:07.072012 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:07.076085 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:59:07.278786 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:07.287276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-23aafe0e-bc0e-4bbb-8134-3ab10b185e8b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1712 microversion: 2.1 time: 0.228886 Sep 30 09:59:07.287847 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 538/1085] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:07 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1712 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:39.411870 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:39.412442 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:39.414567 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:59:39.415238 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=14,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:59:39.419834 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:39.420320 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:39.420755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:39.421446 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:39.421913 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:39.422337 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:39.500260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lazy-loading 'services' on Instance uuid 8872f2fa-9f26-40d0-b256-f8f31dd116e1 {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 09:59:39.693877 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-bb759254-6b0f-4549-a2bf-e2834f3e0eb3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/os-volume_attachments/ad9d918a-ca58-4a22-9731-f82ed6598509" status: 202 len: 0 microversion: 2.1 time: 0.286433 Sep 30 09:59:39.694471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 548/1086] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 09:59:39 2022] DELETE /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/os-volume_attachments/ad9d918a-ca58-4a22-9731-f82ed6598509 => generated 0 bytes in 288 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:59:40.818206 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-978358a2-7bfe-4047-bd0e-c90b1b7e1abf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:59:40.820442 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=43,nova_api:DELETE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:59:40.821075 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=4 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:59:40.822164 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 30 09:59:40.822291 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Fri Sep 30 09:59:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action (ip 10.222.0.39) !!! Sep 30 09:59:40.833066 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-978358a2-7bfe-4047-bd0e-c90b1b7e1abf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:40.833542 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-978358a2-7bfe-4047-bd0e-c90b1b7e1abf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:40.833968 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-978358a2-7bfe-4047-bd0e-c90b1b7e1abf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:40.894449 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-978358a2-7bfe-4047-bd0e-c90b1b7e1abf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Going to try to stop instance {{(pid=127455) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 30 09:59:41.115197 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-978358a2-7bfe-4047-bd0e-c90b1b7e1abf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action" status: 202 len: 0 microversion: 2.1 time: 0.301091 Sep 30 09:59:41.115808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 539/1087] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:59:40 2022] POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action => generated 0 bytes in 303 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:59:41.124410 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:41.125261 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:41.133615 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:41.134103 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:41.134539 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:41.135299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:41.135751 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:41.136174 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:41.267993 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:41.275629 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dbe02678-0441-484e-b7ca-e86c83f5231c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.155477 Sep 30 09:59:41.276187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 549/1088] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:41 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:59:42.299850 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:42.300555 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:42.306826 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:42.307248 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:42.307683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:42.308368 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:42.308817 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:42.309236 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:42.437546 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:42.445603 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-ed5489b4-baa3-498f-b8ca-c21a166a8453 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.150543 Sep 30 09:59:42.446180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 540/1089] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:42 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:43.469217 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:43.469780 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:43.476787 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:43.477270 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:43.477699 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:43.478384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:43.478859 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:43.479282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:43.607231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:43.614919 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7b5dda20-e037-4516-84d8-b357df7ceaa5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.149545 Sep 30 09:59:43.615492 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 550/1090] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:43 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:44.638074 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:44.638708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:44.645256 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:44.645736 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:44.646168 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:44.646898 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:44.647351 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:44.647775 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:44.773441 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:44.781021 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-30e57ae3-0351-4f13-9d97-8266d06afa57 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.146795 Sep 30 09:59:44.781583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 541/1091] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:44 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:45.805656 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:45.806268 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:45.813036 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:45.813520 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:45.813958 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:45.815293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:45.815758 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:45.816186 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:46.175604 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:46.183246 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ecc77654-8bbe-4d02-8835-944386f6ecf2 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.381444 Sep 30 09:59:46.183806 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 551/1092] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:45 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:47.205692 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:47.206295 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:47.212652 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:47.213128 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:47.213549 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:47.214224 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:47.214698 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:47.215124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:47.339271 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:47.346884 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-2d899add-fb34-4cb6-ae49-1eafe1faef3d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.145042 Sep 30 09:59:47.347437 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 542/1093] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:47 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:48.370850 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:48.371467 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:48.378576 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:48.379087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:48.379526 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:48.380221 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:48.380679 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:48.381107 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:48.510149 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:48.518367 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-62b29968-115e-404a-89c4-e5205d92ddde tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.150826 Sep 30 09:59:48.518961 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 552/1094] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:48 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:49.541307 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:49.541924 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:49.548628 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:49.549118 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:49.549558 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:49.550250 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:49.550745 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:49.551205 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:49.680101 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:49.683213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a792199f-b4e4-4e0c-9549-3e3bb7c462ab tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.145795 Sep 30 09:59:49.683778 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 543/1095] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:49 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:59:50.705328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:50.705944 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:50.712636 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:50.713129 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:50.713570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:50.714274 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:50.714768 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:50.715198 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:50.843495 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:50.851658 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e81e8e3e-668b-402f-84ae-c3b2d6587d53 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1676 microversion: 2.1 time: 0.150247 Sep 30 09:59:50.852227 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 553/1096] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:50 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1676 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 09:59:51.874371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:51.874947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:51.882220 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:51.882737 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:51.883165 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:51.883846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:51.884292 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:51.884709 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:52.009793 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:52.017325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1a13a65e-adb1-4e19-bb7b-94e5f07c21f1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1653 microversion: 2.1 time: 0.146844 Sep 30 09:59:52.017885 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 544/1097] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:51 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1653 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:52.040349 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-435b3bb6-1949-4f56-ba52-cce0870bfb31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 09:59:52.046810 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-435b3bb6-1949-4f56-ba52-cce0870bfb31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:52.047293 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-435b3bb6-1949-4f56-ba52-cce0870bfb31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:52.047730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-435b3bb6-1949-4f56-ba52-cce0870bfb31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:52.069748 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-435b3bb6-1949-4f56-ba52-cce0870bfb31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Going to try to start instance {{(pid=127456) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 30 09:59:52.251730 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-435b3bb6-1949-4f56-ba52-cce0870bfb31 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action" status: 202 len: 0 microversion: 2.1 time: 0.215022 Sep 30 09:59:52.252351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 554/1098] 10.222.0.39 () {60 vars in 1278 bytes} [Fri Sep 30 09:59:52 2022] POST /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/action => generated 0 bytes in 217 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 09:59:52.260750 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:52.261362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:52.267305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:52.267809 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:52.268227 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:52.268946 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:52.269401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:52.269826 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:52.401675 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:52.410111 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-c31744eb-0ae1-4df1-beb5-12ece4354dcf tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1662 microversion: 2.1 time: 0.152823 Sep 30 09:59:52.410858 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 545/1099] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:52 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1662 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:53.434078 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:53.434805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:53.440892 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:53.441379 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:53.441815 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:53.442513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:53.442992 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:53.443416 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:53.579597 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:53.587921 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-0c568d4b-2cf8-4b7e-aa7f-e21497768933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1662 microversion: 2.1 time: 0.158427 Sep 30 09:59:53.588675 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 555/1100] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:53 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1662 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 09:59:54.611721 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 09:59:54.612312 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 09:59:54.619245 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:54.619723 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:54.620151 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:54.620825 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 09:59:54.621270 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 09:59:54.621683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 09:59:54.749131 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 09:59:54.757676 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4efad483-0c44-44b8-aaa9-3c9ec84f4bc9 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1666 microversion: 2.1 time: 0.149897 Sep 30 09:59:54.758228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 546/1101] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 09:59:54 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1666 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:27.022032 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:27.024339 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:27.026728 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=16 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:00:27.027399 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=55,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:00:27.032051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:27.035844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:27.035844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:27.035844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:27.035844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:27.036256 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:27.063731 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9ed00e13-94b9-44b8-89d1-e870626ca776 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045563 Sep 30 10:00:27.064516 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 556/1102] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 10:00:27 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 10:00:27.126523 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:27.127055 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:27.129930 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=16 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:00:27.130610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:00:27.135477 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:27.135958 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:27.136401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:27.137080 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:27.137530 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:27.137949 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:27.163285 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Going to try to terminate instance {{(pid=127455) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 10:00:27.378732 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-6964ba0d-d976-4312-9212-e74cb97db8d6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 204 len: 0 microversion: 2.1 time: 0.255682 Sep 30 10:00:27.379319 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 547/1103] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 10:00:27 2022] DELETE /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 10:00:27.388012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:27.395429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:27.395429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:27.395429 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:27.395899 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:27.396473 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:27.396924 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:27.397343 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:27.526230 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:27.534743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ae12a4ae-a76d-4520-9001-613d948cb3f5 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1672 microversion: 2.1 time: 0.150879 Sep 30 10:00:27.535874 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 557/1104] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:27 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1672 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:28.559409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:28.560108 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:28.567297 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:28.567777 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:28.568209 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:28.568893 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:28.569346 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:28.569765 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:28.695492 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:28.703633 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0e1b11ed-3375-4f68-909a-a684bfed30b1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1672 microversion: 2.1 time: 0.148426 Sep 30 10:00:28.704219 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 548/1105] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:28 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1672 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:29.727204 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:29.727852 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:29.734321 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:29.734843 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:29.735282 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:29.735977 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:29.736427 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:29.736853 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:29.876226 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:29.884661 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-4d1d04a2-0624-4cb0-a40a-f8ccc99cb495 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1672 microversion: 2.1 time: 0.161376 Sep 30 10:00:29.885288 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 558/1106] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:29 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1672 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:30.206725 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-64ba37e5-3fd0-430b-b481-e0014bc3a084 req-a4388fce-ade0-462f-8b05-bffdcf67382e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8872f2fa-9f26-40d0-b256-f8f31dd116e1", "name": "network-vif-deleted", "tag": "8354001a-2686-4424-85b3-eda11d52c3b4"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:00:30.216371 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-64ba37e5-3fd0-430b-b481-e0014bc3a084 req-a4388fce-ade0-462f-8b05-bffdcf67382e service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:30.221880 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-64ba37e5-3fd0-430b-b481-e0014bc3a084 req-a4388fce-ade0-462f-8b05-bffdcf67382e service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:30.221880 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-64ba37e5-3fd0-430b-b481-e0014bc3a084 req-a4388fce-ade0-462f-8b05-bffdcf67382e service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:30.228588 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-64ba37e5-3fd0-430b-b481-e0014bc3a084 req-a4388fce-ade0-462f-8b05-bffdcf67382e service nova] Creating event network-vif-deleted:8354001a-2686-4424-85b3-eda11d52c3b4 for instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 on hv-cinder-maste Sep 30 10:00:30.233431 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-64ba37e5-3fd0-430b-b481-e0014bc3a084 req-a4388fce-ade0-462f-8b05-bffdcf67382e service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030373 Sep 30 10:00:30.234221 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 549/1107] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 10:00:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:00:30.908439 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:30.909209 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:30.916551 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:30.917033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:30.917472 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:30.918175 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:30.918720 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:30.919180 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:30.951529 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:30.952026 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:30.952469 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:30.952930 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.objects.instance [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lazy-loading 'fault' on Instance uuid 8872f2fa-9f26-40d0-b256-f8f31dd116e1 {{(pid=127456) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 10:00:30.998111 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:31.006247 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-c8101ca7-4d62-4012-b5f8-d5dca340313c tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 200 len: 1327 microversion: 2.1 time: 0.101817 Sep 30 10:00:31.007113 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 559/1108] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:30 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 1327 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:32.030786 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:32.031482 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:32.038155 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:32.038649 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:32.039081 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:32.039761 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:32.040205 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:32.040619 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:32.053454 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] HTTP exception thrown: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Request method failure captured: Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 HTTP/1.1 Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNr2e8bxOB2Gid_o3E71XGKgrGTCHjEVQp2IwJx-4i3hm874GWlxTTb9RoBpcX4147gkd-kar4QMxPM11mFB6Ft3jgopxk3nOVjLyg2Ao0HBxQErZdlk2RFzt69j0bNRcUrBVfzMmNSafkMI1g23540EdiWVQJyqNPzOCie8LoB4PiU8 Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 10:00:32.055083 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: ee46fc4dd13143bfb409fb5033d9a4d2 Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:00:32.056934 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '8872f2fa-9f26-40d0-b256-f8f31dd116e1'} Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:00:32.058238 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:00:32.060088 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:00:32.061636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Returning 404 to user: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:00:32.061636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-7636e7a6-4ecb-4e4d-8ffa-4fffc409c933 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 404 len: 111 microversion: 2.1 time: 0.030343 Sep 30 10:00:32.061636 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 550/1109] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:32 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 10:00:32.065220 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-6d05b78c-e40c-4cb2-9af2-609121d31f9a tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '.version_select at 0x7f72c0c77040>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:32.133850 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-6d05b78c-e40c-4cb2-9af2-609121d31f9a tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1308355648" status: 202 len: 0 microversion: 2.1 time: 0.072457 Sep 30 10:00:32.134417 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 560/1110] 10.222.0.39 () {58 vars in 1221 bytes} [Fri Sep 30 10:00:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1308355648 => generated 0 bytes in 74 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 10:00:32.827527 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-9bf5c5f3-4292-41e0-a97d-34cfe7c5e677 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'create', calling method: .version_select at 0x7f72c0d134c0>, body: {"keypair": {"name": "tempest-keypair-161004239", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBORxQgSvYPjoJzNo6DUmRhxE+GfmAvWZkgFwe8gFkDDnC7oyCrbMBiEMsJIY3bzTsbqUu/9STB80HrvoHvOLklsgbNKlZbhiNS26mXSOhNowYly3Kb0dG+o+rEWsIFYnEw=="}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:00:32.842651 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-9bf5c5f3-4292-41e0-a97d-34cfe7c5e677 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Getting quotas for project 85eef8fd790146cfa7da6214c13ca262. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 10:00:32.846453 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.quota [None req-9bf5c5f3-4292-41e0-a97d-34cfe7c5e677 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Getting quotas for user ee46fc4dd13143bfb409fb5033d9a4d2 and project 85eef8fd790146cfa7da6214c13ca262. Resources: {'key_pairs'} {{(pid=127455) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 10:00:32.917158 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-9bf5c5f3-4292-41e0-a97d-34cfe7c5e677 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.093453 Sep 30 10:00:32.917867 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 551/1111] 10.222.0.39 () {60 vars in 1159 bytes} [Fri Sep 30 10:00:32 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:00:34.921257 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1245566166", "imageRef": "1c24d34a-e765-46c5-be1a-904ccbc2d671", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6c765eee-bae8-4b34-897a-abb29acd5a27"}], "security_groups": [{"name": "tempest-securitygroup--1715404644"}], "key_name": "tempest-keypair-161004239", "user_data": "}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:00:35.022362 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.network.neutron [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] validate_networks() for [('6c765eee-bae8-4b34-897a-abb29acd5a27', None, None, None, None, None)] {{(pid=127456) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 30 10:00:35.385460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Flavor limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 30 10:00:35.385875 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Image limits 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 30 10:00:35.386343 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Flavor pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 30 10:00:35.386780 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Image pref 0:0:0 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 30 10:00:35.387187 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.virt.hardware [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127456) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 30 10:00:35.396018 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:35.399351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:35.399351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:35.399351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:35.399351 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:35.399723 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:35.416072 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Getting quotas for project 85eef8fd790146cfa7da6214c13ca262. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 30 10:00:35.421012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.quota [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Getting quotas for user ee46fc4dd13143bfb409fb5033d9a4d2 and project 85eef8fd790146cfa7da6214c13ca262. Resources: {'ram', 'instances', 'cores'} {{(pid=127456) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 30 10:00:35.432531 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Going to run 1 instances... {{(pid=127456) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 30 10:00:35.434834 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1c24d34a-e765-46c5-be1a-904ccbc2d671',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127456) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 30 10:00:35.517551 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-46a179a0-e508-44bd-9a88-c3c408d80c36 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.601065 Sep 30 10:00:35.518192 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 561/1112] 10.222.0.39 () {60 vars in 1147 bytes} [Fri Sep 30 10:00:34 2022] POST /compute/v2.1/servers => generated 402 bytes in 602 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 30 10:00:35.529704 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-359598eb-d866-4403-af11-3657be84e45b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:35.530316 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-359598eb-d866-4403-af11-3657be84e45b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:35.585465 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-359598eb-d866-4403-af11-3657be84e45b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:35.592965 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-359598eb-d866-4403-af11-3657be84e45b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1237 microversion: 2.1 time: 0.067588 Sep 30 10:00:35.593607 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 552/1113] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:35 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1237 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:36.616755 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:36.617398 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:36.623824 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:36.624319 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:36.624752 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:36.625445 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:36.625894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:36.626323 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:36.689568 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:36.697917 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5c8fae7d-3fcc-4213-bb06-0d0bbd9488bd tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1233 microversion: 2.1 time: 0.086380 Sep 30 10:00:36.706005 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 562/1114] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:36 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1233 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:37.720390 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:37.720968 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:37.728031 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:37.728526 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:37.728960 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:37.729663 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:37.730117 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:37.730543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:37.795389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:37.803147 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-71425d4c-7ecd-443e-be7d-c80de229d71b tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1295 microversion: 2.1 time: 0.086704 Sep 30 10:00:37.803839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 553/1115] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:37 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1295 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:38.826566 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:38.827331 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:38.834084 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:38.834507 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:38.834983 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:38.835671 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:38.836118 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:38.836539 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:38.968779 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:38.977141 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-aa2d3b3f-a332-4156-a0b7-128e81f51f09 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1363 microversion: 2.1 time: 0.154496 Sep 30 10:00:38.977692 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 563/1116] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:38 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:39.024785 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-a4661e99-a8eb-4f97-86d2-44c39d47c80b req-8b179b26-b7bd-4b4c-be61-e7a9c52db849 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9ab42327-a15b-4fd8-ac92-a2334ed2c107", "tag": "5a8a2d85-a9ed-4cda-a769-17f985851fae"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:00:39.034707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-a4661e99-a8eb-4f97-86d2-44c39d47c80b req-8b179b26-b7bd-4b4c-be61-e7a9c52db849 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:39.035178 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-a4661e99-a8eb-4f97-86d2-44c39d47c80b req-8b179b26-b7bd-4b4c-be61-e7a9c52db849 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:39.036720 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-a4661e99-a8eb-4f97-86d2-44c39d47c80b req-8b179b26-b7bd-4b4c-be61-e7a9c52db849 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:39.047450 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-a4661e99-a8eb-4f97-86d2-44c39d47c80b req-8b179b26-b7bd-4b4c-be61-e7a9c52db849 service nova] Creating event network-changed:5a8a2d85-a9ed-4cda-a769-17f985851fae for instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 on hv-cinder-maste Sep 30 10:00:39.052967 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-a4661e99-a8eb-4f97-86d2-44c39d47c80b req-8b179b26-b7bd-4b4c-be61-e7a9c52db849 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031781 Sep 30 10:00:39.053654 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 554/1117] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 10:00:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:00:39.999586 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:40.000171 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:40.006513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:40.007223 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:40.007656 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:40.008333 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:40.008784 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:40.009203 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:40.133940 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:40.141448 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e629d71d-2e4e-4bdb-95bd-4777b77b3fdb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1363 microversion: 2.1 time: 0.145754 Sep 30 10:00:40.142012 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 564/1118] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:39 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1363 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:41.164565 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:41.165213 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:41.171333 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:41.171812 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:41.172241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:41.172959 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:41.173416 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:41.173842 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:41.307584 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:41.315610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-0fb90703-d17f-4d2b-bab5-de9ddb4cde82 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1363 microversion: 2.1 time: 0.155810 Sep 30 10:00:41.316171 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 555/1119] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:41 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:42.340860 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:42.341397 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:42.347819 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:42.348326 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:42.348762 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:42.349442 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:42.349893 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:42.350310 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:42.476013 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:42.483614 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-8090c161-eb95-4273-ab90-885b33a56ba3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1363 microversion: 2.1 time: 0.148121 Sep 30 10:00:42.484181 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 565/1120] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:42 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1363 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:43.507114 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:43.507803 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:43.516094 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:43.516570 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:43.516993 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:43.517682 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:43.518132 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:43.518550 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:43.646248 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:43.654441 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-b8316f97-9e6c-4c3b-a8c0-cea71f696994 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1363 microversion: 2.1 time: 0.151654 Sep 30 10:00:43.655026 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 556/1121] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:43 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1363 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:44.677262 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:44.677905 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:44.683809 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:44.684290 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:44.684719 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:44.685439 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:44.685895 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:44.686318 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:44.814929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:44.822468 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-444d90cd-b022-4a04-9093-9bb6475037ca tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1363 microversion: 2.1 time: 0.149329 Sep 30 10:00:44.823077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 566/1122] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:44 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1363 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:45.509415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-3833e2aa-b209-4e65-936b-219b68fd4f63 req-c50c26ce-0aed-4988-92d5-61e4ed3048f6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9ab42327-a15b-4fd8-ac92-a2334ed2c107", "name": "network-vif-plugged", "status": "completed", "tag": "5a8a2d85-a9ed-4cda-a769-17f985851fae"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:00:45.519896 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3833e2aa-b209-4e65-936b-219b68fd4f63 req-c50c26ce-0aed-4988-92d5-61e4ed3048f6 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:45.520597 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3833e2aa-b209-4e65-936b-219b68fd4f63 req-c50c26ce-0aed-4988-92d5-61e4ed3048f6 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:45.521020 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-3833e2aa-b209-4e65-936b-219b68fd4f63 req-c50c26ce-0aed-4988-92d5-61e4ed3048f6 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:45.532010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-3833e2aa-b209-4e65-936b-219b68fd4f63 req-c50c26ce-0aed-4988-92d5-61e4ed3048f6 service nova] Creating event network-vif-plugged:5a8a2d85-a9ed-4cda-a769-17f985851fae for instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 on hv-cinder-maste Sep 30 10:00:45.536947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-3833e2aa-b209-4e65-936b-219b68fd4f63 req-c50c26ce-0aed-4988-92d5-61e4ed3048f6 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031544 Sep 30 10:00:45.537524 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 557/1123] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 10:00:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:00:45.845544 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:45.846148 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:45.848009 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=2 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:00:45.848733 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell0:SELECT=6 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:00:45.854133 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:45.855125 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:45.855579 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:45.856295 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:45.856743 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:45.857163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:45.986366 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:45.993951 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-dd8eeaaa-3e61-44f9-ba9a-1d576029967e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1363 microversion: 2.1 time: 0.152544 Sep 30 10:00:45.994513 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 567/1124] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:45 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1363 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:47.014839 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell0:SELECT=6 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:00:47.019481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:47.020181 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:47.026200 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:47.027660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:47.027660 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:47.027903 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:47.028311 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:47.028732 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:47.160042 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:00:47.167563 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-66561204-66d9-4e86-8df3-0088929932ce tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1548 microversion: 2.1 time: 0.153704 Sep 30 10:00:47.168130 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 558/1125] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:00:47 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1548 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:00:47.190357 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:00:47.190916 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:00:47.197245 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:47.197734 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:47.198179 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:47.199610 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:47.200311 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:47.200476 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:47.278777 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-cd15fbe5-9d1a-470f-99d0-597578cc9100 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092231 Sep 30 10:00:47.279672 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 568/1126] 10.222.0.39 () {58 vars in 1275 bytes} [Fri Sep 30 10:00:47 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-interface => generated 284 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:00:48.254163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [req-00c34b2b-a489-42a0-83e4-53c94dd4316b req-fc45bb0b-d9e9-4e9d-bd7b-5809a5e52497 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9ab42327-a15b-4fd8-ac92-a2334ed2c107", "tag": "5a8a2d85-a9ed-4cda-a769-17f985851fae"}]} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:00:48.264346 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-00c34b2b-a489-42a0-83e4-53c94dd4316b req-fc45bb0b-d9e9-4e9d-bd7b-5809a5e52497 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:00:48.264813 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-00c34b2b-a489-42a0-83e4-53c94dd4316b req-fc45bb0b-d9e9-4e9d-bd7b-5809a5e52497 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:00:48.265235 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [req-00c34b2b-a489-42a0-83e4-53c94dd4316b req-fc45bb0b-d9e9-4e9d-bd7b-5809a5e52497 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:00:48.275947 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.compute.server_external_events [req-00c34b2b-a489-42a0-83e4-53c94dd4316b req-fc45bb0b-d9e9-4e9d-bd7b-5809a5e52497 service nova] Creating event network-changed:5a8a2d85-a9ed-4cda-a769-17f985851fae for instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 on hv-cinder-maste Sep 30 10:00:48.284459 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [req-00c34b2b-a489-42a0-83e4-53c94dd4316b req-fc45bb0b-d9e9-4e9d-bd7b-5809a5e52497 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030236 Sep 30 10:00:48.284459 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 559/1127] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 10:00:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:24.354701 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_api:SELECT=42,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:01:24.355485 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG dbcounter [-] [127456] Writing DB stats nova_cell1:SELECT=68 {{(pid=127456) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:01:24.359713 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:24.360230 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:24.366317 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:24.366831 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:24.367267 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:24.367953 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:24.368406 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:24.368828 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:24.393638 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-d3372553-fe74-4f7f-b71c-2f565f157846 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/ips" status: 200 len: 145 microversion: 2.1 time: 0.040571 Sep 30 10:01:24.394202 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 569/1128] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 10:01:24 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/ips => generated 145 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:25.890610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:01:25.891462 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:01:25.896097 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "00ee38ab-9768-457e-a331-f42ca89d839f"}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:01:25.900708 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:25.904676 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:25.905150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:25.905573 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:25.906251 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:25.906733 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:25.907150 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:26.753036 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-e27640e4-561d-49a1-923f-c9ef6f622c46 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.863842 Sep 30 10:01:26.754683 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 560/1129] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 10:01:25 2022] POST /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments => generated 194 bytes in 866 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:34.278156 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:34.280478 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:34.288252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:34.288734 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:34.289163 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:34.289849 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:34.290299 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:34.290754 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:34.320922 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-75cb91a6-77b3-4352-b9a2-15b29fd4f6a6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046754 Sep 30 10:01:34.321461 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 570/1130] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 10:01:34 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:34.333230 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:34.333808 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:34.340305 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:34.340782 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:34.341205 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:34.341890 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:34.342325 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:34.342772 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:34.370620 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d54bfff2-2bd1-4783-ad22-2c3072697abb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/00ee38ab-9768-457e-a331-f42ca89d839f" status: 200 len: 194 microversion: 2.1 time: 0.041628 Sep 30 10:01:34.371180 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 561/1131] 10.222.0.39 () {58 vars in 1413 bytes} [Fri Sep 30 10:01:34 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/00ee38ab-9768-457e-a331-f42ca89d839f => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:35.970837 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "88bf9c67-6833-49d9-ac6d-6f88e9321dbe"}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:01:35.973209 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:35.980137 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:35.980643 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:35.981081 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:35.981757 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:35.982207 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:35.982648 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:36.638551 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-1b08f6e9-30bf-4b26-96ce-8ad52895dfeb tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "POST /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.671487 Sep 30 10:01:36.639323 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 571/1132] 10.222.0.39 () {60 vars in 1323 bytes} [Fri Sep 30 10:01:35 2022] POST /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments => generated 194 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:44.171459 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:44.173747 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:44.180501 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:44.181010 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:44.181456 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:44.182142 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:44.182625 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:44.183610 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:44.213575 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-03c338b9-f0c7-4e63-9526-3136df41045f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.046536 Sep 30 10:01:44.214143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 562/1133] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 10:01:44 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments => generated 371 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:44.357727 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:44.358285 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:44.364353 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:44.364844 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:44.365284 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:44.365981 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:44.366440 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:44.366895 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:44.395560 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-67c64004-9cfb-454e-adaf-b7bfe4157dc6 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/00ee38ab-9768-457e-a331-f42ca89d839f" status: 200 len: 194 microversion: 2.1 time: 0.043304 Sep 30 10:01:44.396139 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 572/1134] 10.222.0.39 () {58 vars in 1413 bytes} [Fri Sep 30 10:01:44 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/00ee38ab-9768-457e-a331-f42ca89d839f => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:44.406465 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:44.407003 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:44.412991 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:44.413475 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:44.413910 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:44.414680 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:44.415141 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:44.415563 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:44.497411 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lazy-loading 'services' on Instance uuid 9ab42327-a15b-4fd8-ac92-a2334ed2c107 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 10:01:44.674607 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-84244098-1658-406a-82e6-d8bc3386fbd8 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/00ee38ab-9768-457e-a331-f42ca89d839f" status: 202 len: 0 microversion: 2.1 time: 0.271929 Sep 30 10:01:44.675240 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 563/1135] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 10:01:44 2022] DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/00ee38ab-9768-457e-a331-f42ca89d839f => generated 0 bytes in 273 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 10:01:46.849532 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:46.850033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:46.856170 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:46.856652 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:46.857082 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:46.857766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:46.858219 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:46.858668 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:46.887167 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-69c521dd-1e55-4da1-9b26-47a08ec67131 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/88bf9c67-6833-49d9-ac6d-6f88e9321dbe" status: 200 len: 194 microversion: 2.1 time: 0.041487 Sep 30 10:01:46.888308 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 573/1136] 10.222.0.39 () {58 vars in 1413 bytes} [Fri Sep 30 10:01:46 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/88bf9c67-6833-49d9-ac6d-6f88e9321dbe => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:46.899334 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:46.899943 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:46.906523 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:46.907040 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:46.907481 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:46.908167 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:46.908620 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:46.909041 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:46.987104 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lazy-loading 'services' on Instance uuid 9ab42327-a15b-4fd8-ac92-a2334ed2c107 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 10:01:47.158833 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1df7927a-2a1d-4127-bf98-9e12a551cff1 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/88bf9c67-6833-49d9-ac6d-6f88e9321dbe" status: 202 len: 0 microversion: 2.1 time: 0.263553 Sep 30 10:01:47.159419 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 564/1137] 10.222.0.39 () {58 vars in 1416 bytes} [Fri Sep 30 10:01:46 2022] DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments/88bf9c67-6833-49d9-ac6d-6f88e9321dbe => generated 0 bytes in 265 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 10:01:48.339764 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:48.342708 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:48.348963 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:48.349444 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:48.349876 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:48.350565 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:48.351039 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:48.351457 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:48.379573 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-e8bb5490-fdff-4c39-8618-64a9b06d575f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043869 Sep 30 10:01:48.380122 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 574/1138] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 10:01:48 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 10:01:48.493988 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:48.496290 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:48.502914 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:48.503402 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:48.503840 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:48.504534 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:48.504986 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:48.505409 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:48.534058 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-55d898e3-9c70-4532-aa70-990ebaa9fb5e tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044894 Sep 30 10:01:48.534615 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 565/1139] 10.222.0.39 () {58 vars in 1302 bytes} [Fri Sep 30 10:01:48 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 30 10:01:48.595965 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:48.596403 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:48.602225 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:48.602739 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:48.603172 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:48.603856 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:48.604306 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:48.604725 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:48.629596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Going to try to terminate instance {{(pid=127456) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 30 10:01:48.851306 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-b64b1a05-97f5-4c87-b2fb-49a6d7f7134d tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 204 len: 0 microversion: 2.1 time: 0.258844 Sep 30 10:01:48.851823 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 575/1140] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 10:01:48 2022] DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 30 10:01:48.860511 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:48.861112 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:48.866933 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:48.867410 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:48.867834 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:48.868516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:48.868960 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:48.869453 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:48.999389 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:01:49.007203 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-f5e3d6e3-9bf1-4300-b127-f4a9f5b270de tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1673 microversion: 2.1 time: 0.150828 Sep 30 10:01:49.007762 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 566/1141] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:01:48 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1673 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:01:50.082355 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:50.083077 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:50.089384 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:50.089868 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:50.090303 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:50.091033 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:50.091489 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:50.091912 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:50.219024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.policy [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127456) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:01:50.226677 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-2fe0617d-60fb-457f-ae00-aa35d9b4b790 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1673 microversion: 2.1 time: 0.148604 Sep 30 10:01:50.227239 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 576/1142] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:01:50 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1673 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 30 10:01:51.250289 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:51.250955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:51.257916 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:51.258436 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:51.258905 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:51.259622 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:51.260077 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:51.260505 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:51.401143 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:01:51.409350 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-79f7bdf5-8261-4589-b24b-ab5f5cebdb07 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1673 microversion: 2.1 time: 0.163366 Sep 30 10:01:51.409945 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 567/1143] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:01:51 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1673 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 10:01:51.920780 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [req-4e11461f-973f-482d-a119-734e8763eda1 req-36fe5c0e-5fe1-4c77-beaf-7fde766d9d48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9ab42327-a15b-4fd8-ac92-a2334ed2c107", "name": "network-vif-deleted", "tag": "5a8a2d85-a9ed-4cda-a769-17f985851fae"}]} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:01:51.931243 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-4e11461f-973f-482d-a119-734e8763eda1 req-36fe5c0e-5fe1-4c77-beaf-7fde766d9d48 service nova] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:51.931764 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-4e11461f-973f-482d-a119-734e8763eda1 req-36fe5c0e-5fe1-4c77-beaf-7fde766d9d48 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:51.932195 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [req-4e11461f-973f-482d-a119-734e8763eda1 req-36fe5c0e-5fe1-4c77-beaf-7fde766d9d48 service nova] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:51.944215 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.compute.server_external_events [req-4e11461f-973f-482d-a119-734e8763eda1 req-36fe5c0e-5fe1-4c77-beaf-7fde766d9d48 service nova] Creating event network-vif-deleted:5a8a2d85-a9ed-4cda-a769-17f985851fae for instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 on hv-cinder-maste Sep 30 10:01:51.949229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [req-4e11461f-973f-482d-a119-734e8763eda1 req-36fe5c0e-5fe1-4c77-beaf-7fde766d9d48 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032717 Sep 30 10:01:51.949878 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 577/1144] 10.222.0.39 () {64 vars in 1317 bytes} [Fri Sep 30 10:01:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 30 10:01:52.433500 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:52.434314 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:52.440735 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:52.441140 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:52.441577 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:52.442277 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:52.442774 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:52.443228 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:52.474599 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:52.475104 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:52.475539 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:52.475984 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.objects.instance [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lazy-loading 'fault' on Instance uuid 9ab42327-a15b-4fd8-ac92-a2334ed2c107 {{(pid=127455) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 30 10:01:52.516013 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.policy [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee46fc4dd13143bfb409fb5033d9a4d2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85eef8fd790146cfa7da6214c13ca262', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127455) authorize /opt/stack/nova/nova/policy.py:199}} Sep 30 10:01:52.523704 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-87c0d140-e947-4e46-bdd7-65e48f2e09e0 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 200 len: 1327 microversion: 2.1 time: 0.094231 Sep 30 10:01:52.524276 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 568/1145] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:01:52 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 1327 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 30 10:01:53.548348 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:53.549039 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:53.555087 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:53.555567 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:53.556024 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:53.556706 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:53.557155 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:53.557572 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:53.570344 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] HTTP exception thrown: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Request method failure captured: Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 HTTP/1.1 Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNr2e8bxOB2Gid_o3E71XGKgrGTCHjEVQp2IwJx-4i3hm874GWlxTTb9RoBpcX4147gkd-kar4QMxPM11mFB6Ft3jgopxk3nOVjLyg2Ao0HBxQErZdlk2RFzt69j0bNRcUrBVfzMmNSafkMI1g23540EdiWVQJyqNPzOCie8LoB4PiU8 Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 10:01:53.571933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: ee46fc4dd13143bfb409fb5033d9a4d2 Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:53.573766 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '9ab42327-a15b-4fd8-ac92-a2334ed2c107'} Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:01:53.574805 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:53.576460 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:53.577967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Returning 404 to user: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:01:53.577967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ad13de17-1cfa-4332-964a-01e2a47feb23 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 404 len: 111 microversion: 2.1 time: 0.030265 Sep 30 10:01:53.577967 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 578/1146] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:01:53 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 10:01:53.581901 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-a99b4332-1336-4cf2-a1dc-570d572ceeea tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '.version_select at 0x7f72c0e99e50>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:53.663804 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-a99b4332-1336-4cf2-a1dc-570d572ceeea tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-161004239" status: 202 len: 0 microversion: 2.1 time: 0.085602 Sep 30 10:01:53.664516 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 569/1147] 10.222.0.39 () {58 vars in 1218 bytes} [Fri Sep 30 10:01:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-161004239 => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 30 10:01:56.850120 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:56.850646 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:56.857412 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:56.857894 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:56.858329 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:56.859051 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:56.859505 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:56.859929 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:56.873774 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] HTTP exception thrown: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Request method failure captured: Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 HTTP/1.1 Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNr2e8bxOB2Gid_o3E71XGKgrGTCHjEVQp2IwJx-4i3hm874GWlxTTb9RoBpcX4147gkd-kar4QMxPM11mFB6Ft3jgopxk3nOVjLyg2Ao0HBxQErZdlk2RFzt69j0bNRcUrBVfzMmNSafkMI1g23540EdiWVQJyqNPzOCie8LoB4PiU8 Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 10:01:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: ee46fc4dd13143bfb409fb5033d9a4d2 Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.877252 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '9ab42327-a15b-4fd8-ac92-a2334ed2c107'} Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:01:56.878248 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.879884 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.881374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Returning 404 to user: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:01:56.881374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-5938efb3-eaeb-4c9b-8ad4-e9a8ef854d2f tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 404 len: 111 microversion: 2.1 time: 0.030584 Sep 30 10:01:56.881374 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 579/1148] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 10:01:56 2022] DELETE /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 10:01:56.885955 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:56.886567 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 9ab42327-a15b-4fd8-ac92-a2334ed2c107] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:56.890912 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_api:SELECT=24,nova_api:DELETE=1 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:01:56.896241 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:56.896778 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:56.897217 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:56.897904 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:56.898475 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:56.898944 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:56.903415 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG dbcounter [-] [127455] Writing DB stats nova_cell1:SELECT=68,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=127455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:01:56.913659 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] HTTP exception thrown: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Request method failure captured: Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 HTTP/1.1 Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNr2e8bxOB2Gid_o3E71XGKgrGTCHjEVQp2IwJx-4i3hm874GWlxTTb9RoBpcX4147gkd-kar4QMxPM11mFB6Ft3jgopxk3nOVjLyg2Ao0HBxQErZdlk2RFzt69j0bNRcUrBVfzMmNSafkMI1g23540EdiWVQJyqNPzOCie8LoB4PiU8 Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 10:01:56.915401 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: ee46fc4dd13143bfb409fb5033d9a4d2 Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.917304 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '9ab42327-a15b-4fd8-ac92-a2334ed2c107'} Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:01:56.918322 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. Sep 30 10:01:56.920512 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:56.922081 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Returning 404 to user: Instance 9ab42327-a15b-4fd8-ac92-a2334ed2c107 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:01:56.922081 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4f861428-af91-4ad3-a20f-3be09e196df3 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107" status: 404 len: 111 microversion: 2.1 time: 0.035671 Sep 30 10:01:56.922081 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 570/1149] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:01:56 2022] GET /compute/v2.1/servers/9ab42327-a15b-4fd8-ac92-a2334ed2c107 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 10:01:58.170523 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:58.171057 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.compute.api [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127456) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:58.177455 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:58.177933 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:58.178370 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:58.179147 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:58.179596 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:58.180011 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG oslo_concurrency.lockutils [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127456) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:58.193650 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] HTTP exception thrown: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Request method failure captured: Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 HTTP/1.1 Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNr2e8bxOB2Gid_o3E71XGKgrGTCHjEVQp2IwJx-4i3hm874GWlxTTb9RoBpcX4147gkd-kar4QMxPM11mFB6Ft3jgopxk3nOVjLyg2Ao0HBxQErZdlk2RFzt69j0bNRcUrBVfzMmNSafkMI1g23540EdiWVQJyqNPzOCie8LoB4PiU8 Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 10:01:58.195229 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: ee46fc4dd13143bfb409fb5033d9a4d2 Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: > Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.197037 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': '8872f2fa-9f26-40d0-b256-f8f31dd116e1'} Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:01:58.198231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.200328 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.202838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Returning 404 to user: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:01:58.202838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-639edab4-5450-4f6d-ae62-69c958457e81 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "DELETE /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 404 len: 111 microversion: 2.1 time: 0.030069 Sep 30 10:01:58.202838 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 580/1150] 10.222.0.39 () {58 vars in 1239 bytes} [Fri Sep 30 10:01:58 2022] DELETE /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 10:01:58.206184 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Calling method '>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:01:58.206820 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.compute.api [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] [instance: 8872f2fa-9f26-40d0-b256-f8f31dd116e1] Fetching instance by UUID {{(pid=127455) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 30 10:01:58.213941 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:58.214423 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:58.214896 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:58.215583 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Acquiring lock "f6355185-7b23-453c-870b-129ec40cec19" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 30 10:01:58.216033 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 30 10:01:58.216456 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG oslo_concurrency.lockutils [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Lock "f6355185-7b23-453c-870b-129ec40cec19" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127455) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 30 10:01:58.229471 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] HTTP exception thrown: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Request method failure captured: Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 HTTP/1.1 Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNr2e8bxOB2Gid_o3E71XGKgrGTCHjEVQp2IwJx-4i3hm874GWlxTTb9RoBpcX4147gkd-kar4QMxPM11mFB6Ft3jgopxk3nOVjLyg2Ao0HBxQErZdlk2RFzt69j0bNRcUrBVfzMmNSafkMI1g23540EdiWVQJyqNPzOCie8LoB4PiU8 Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 10:01:58.231004 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_85eef8fd790146cfa7da6214c13ca262"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 85eef8fd790146cfa7da6214c13ca262 Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1314467243 Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: ee46fc4dd13143bfb409fb5033d9a4d2 Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-AttachVolumeTestJSON-1314467243-project Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: > Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.232846 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '8872f2fa-9f26-40d0-b256-f8f31dd116e1'} Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:01:58.233792 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. Sep 30 10:01:58.235287 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:01:58.236543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] Returning 404 to user: Instance 8872f2fa-9f26-40d0-b256-f8f31dd116e1 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:01:58.236543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-4b90bbbe-f34c-44e9-8db1-c68834df9643 tempest-AttachVolumeTestJSON-1314467243 tempest-AttachVolumeTestJSON-1314467243-project] 10.222.0.39 "GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1" status: 404 len: 111 microversion: 2.1 time: 0.030595 Sep 30 10:01:58.236543 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 571/1151] 10.222.0.39 () {58 vars in 1236 bytes} [Fri Sep 30 10:01:58 2022] GET /compute/v2.1/servers/8872f2fa-9f26-40d0-b256-f8f31dd116e1 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 10:02:03.200570 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7f6e9d36-a2e3-4485-8422-2497701026ac tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Action: 'create', calling method: .version_select at 0x7f72c0b54c10>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1627865727", "metadata": {"Type": "work"}}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7f6e9d36-a2e3-4485-8422-2497701026ac tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Request method failure captured: Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 122 Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNr6bZR0iYkFHn4Pd-KuHuvDs-GYcugrhhRyCv9u7CVRlsHeO6gfZ-9RaXfArThBpZFxpRyUZuNF0ZnvUJSEdilEQL2jZ7vWyIOVScFWrsDBNNgdiWtriwbsoscO9uKjxB4vVJNX_8MrN54HHEqjDRe_DGwM1H7-K4_tuzaFXAf7_MD8 Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 10:02:03.209231 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/079de817fc624bd4b00f344c664d951c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_079de817fc624bd4b00f344c664d951c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 71898ee085d645bb959bef0e6494561d Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1627865727", "metadata": {"Type": "work"}}} Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: Sep 30 10:02:03.211199 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1627865727', 'metadata': {'Type': 'work'}}}} Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise error Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi '#$%' Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:02:03.212292 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 30 10:02:03.213946 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 30 10:02:03.215681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 30 10:02:03.215681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.215681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-7f6e9d36-a2e3-4485-8422-2497701026ac tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:02:03.215681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-7f6e9d36-a2e3-4485-8422-2497701026ac tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.124479 Sep 30 10:02:03.215681 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 581/1152] 10.222.0.39 () {60 vars in 1156 bytes} [Fri Sep 30 10:02:03 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 126 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 30 10:02:03.221628 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d8935f61-67f8-4536-961e-4cb82143ecb7 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Action: 'create', calling method: .version_select at 0x7f72c0e99e50>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-45462902", "metadata": {"Type": "work"}}} {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d8935f61-67f8-4536-961e-4cb82143ecb7 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Request method failure captured: Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 118 Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNr6bZR0iYkFHn4Pd-KuHuvDs-GYcugrhhRyCv9u7CVRlsHeO6gfZ-9RaXfArThBpZFxpRyUZuNF0ZnvUJSEdilEQL2jZ7vWyIOVScFWrsDBNNgdiWtriwbsoscO9uKjxB4vVJNX_8MrN54HHEqjDRe_DGwM1H7-K4_tuzaFXAf7_MD8 Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 10:02:03.226338 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/079de817fc624bd4b00f344c664d951c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_079de817fc624bd4b00f344c664d951c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 71898ee085d645bb959bef0e6494561d Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-45462902", "metadata": {"Type": "work"}}} Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: Sep 30 10:02:03.228133 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-45462902', 'metadata': {'Type': 'work'}}}} Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise error Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi '0' Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:02:03.229124 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 30 10:02:03.230779 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 30 10:02:03.232771 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 30 10:02:03.232771 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.232771 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-d8935f61-67f8-4536-961e-4cb82143ecb7 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:02:03.232771 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-d8935f61-67f8-4536-961e-4cb82143ecb7 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010350 Sep 30 10:02:03.232771 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 572/1153] 10.222.0.39 () {60 vars in 1156 bytes} [Fri Sep 30 10:02:03 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 30 10:02:03.238153 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ef5af3bf-8b97-41e9-9ac7-3752e6c1b4a7 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Action: 'create', calling method: .version_select at 0x7f72c443b3a0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1811159615", "metadata": {"Type": "work"}}} {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ef5af3bf-8b97-41e9-9ac7-3752e6c1b4a7 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Request method failure captured: Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 119 Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNr6bZR0iYkFHn4Pd-KuHuvDs-GYcugrhhRyCv9u7CVRlsHeO6gfZ-9RaXfArThBpZFxpRyUZuNF0ZnvUJSEdilEQL2jZ7vWyIOVScFWrsDBNNgdiWtriwbsoscO9uKjxB4vVJNX_8MrN54HHEqjDRe_DGwM1H7-K4_tuzaFXAf7_MD8 Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 10:02:03.244260 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/079de817fc624bd4b00f344c664d951c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_079de817fc624bd4b00f344c664d951c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 71898ee085d645bb959bef0e6494561d Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1811159615", "metadata": {"Type": "work"}}} Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: Sep 30 10:02:03.246008 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1811159615', 'metadata': {'Type': 'work'}}}} Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise error Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi '' Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:02:03.247131 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 30 10:02:03.248881 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 30 10:02:03.250641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 30 10:02:03.250641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.250641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-ef5af3bf-8b97-41e9-9ac7-3752e6c1b4a7 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:02:03.250641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-ef5af3bf-8b97-41e9-9ac7-3752e6c1b4a7 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.011357 Sep 30 10:02:03.250641 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 582/1154] 10.222.0.39 () {60 vars in 1156 bytes} [Fri Sep 30 10:02:03 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 30 10:02:03.255522 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1cbc8126-0d8e-4431-9ae7-bdb2074f80ca tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Calling method '.version_select at 0x7f72c0f62f70>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:02:03.284774 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-1cbc8126-0d8e-4431-9ae7-bdb2074f80ca tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] HTTP exception thrown: Volume tempest-invalid-663493763 could not be found. Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1cbc8126-0d8e-4431-9ae7-bdb2074f80ca tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Request method failure captured: Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-663493763 HTTP/1.1 Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNr6bZR0iYkFHn4Pd-KuHuvDs-GYcugrhhRyCv9u7CVRlsHeO6gfZ-9RaXfArThBpZFxpRyUZuNF0ZnvUJSEdilEQL2jZ7vWyIOVScFWrsDBNNgdiWtriwbsoscO9uKjxB4vVJNX_8MrN54HHEqjDRe_DGwM1H7-K4_tuzaFXAf7_MD8 Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 10:02:03.286818 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/079de817fc624bd4b00f344c664d951c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_079de817fc624bd4b00f344c664d951c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 71898ee085d645bb959bef0e6494561d Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Volume tempest-invalid-663493763 could not be found. Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': 'tempest-invalid-663493763'} Sep 30 10:02:03.288564 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-663493763 could not be found. (HTTP 404) (Request-ID: req-19946053-34e5-46d0-8706-150db2d9e847) Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.289653 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 30 10:02:03.291318 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-663493763 could not be found. Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:02:03.292694 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-663493763 could not be found. Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-663493763 could not be found. Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-1cbc8126-0d8e-4431-9ae7-bdb2074f80ca tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Returning 404 to user: Volume tempest-invalid-663493763 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-1cbc8126-0d8e-4431-9ae7-bdb2074f80ca tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/tempest-invalid-663493763" status: 404 len: 98 microversion: 2.1 time: 0.037387 Sep 30 10:02:03.294996 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 573/1155] 10.222.0.39 () {58 vars in 1215 bytes} [Fri Sep 30 10:02:03 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-663493763 => generated 98 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 30 10:02:03.298315 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-9179f9cb-51f4-4e75-bc61-46d7657a39cd tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004346 Sep 30 10:02:03.298841 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 583/1156] 10.222.0.39 () {58 vars in 1140 bytes} [Fri Sep 30 10:02:03 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 30 10:02:03.307680 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-69529567-9079-42c0-bc71-f1d2ab96e678 tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004103 Sep 30 10:02:03.308163 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 574/1157] 10.222.0.39 () {58 vars in 1137 bytes} [Fri Sep 30 10:02:03 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Sep 30 10:02:03.315914 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-934e8a96-1551-47ca-a85d-f765a6ae432f tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Calling method '.version_select at 0x7f72c0930ee0>' {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:02:03.343637 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.wsgi [None req-934e8a96-1551-47ca-a85d-f765a6ae432f tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] HTTP exception thrown: Volume dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 could not be found. Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-934e8a96-1551-47ca-a85d-f765a6ae432f tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Request method failure captured: Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: request: DELETE /compute/v2.1/os-volumes/dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 HTTP/1.1 Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept: application/json Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Accept-Encoding: identity Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Connection: close Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Length: 0 Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Content-Type: application/json Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Host: 10.222.0.39 Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: Openstack-System-Scope: None Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: User-Agent: python-urllib3/1.26.12 Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Auth-Token: gAAAAABjNr6bZR0iYkFHn4Pd-KuHuvDs-GYcugrhhRyCv9u7CVRlsHeO6gfZ-9RaXfArThBpZFxpRyUZuNF0ZnvUJSEdilEQL2jZ7vWyIOVScFWrsDBNNgdiWtriwbsoscO9uKjxB4vVJNX_8MrN54HHEqjDRe_DGwM1H7-K4_tuzaFXAf7_MD8 Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Id: None Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Domain-Name: None Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Identity-Status: Confirmed Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Is-Admin-Project: True Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Id: default Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Domain-Name: Default Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Project-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Role: reader,member Sep 30 10:02:03.345536 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Roles: reader,member Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/079de817fc624bd4b00f344c664d951c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_079de817fc624bd4b00f344c664d951c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-Tenant-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Id: default Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Domain-Name: Default Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Id: 71898ee085d645bb959bef0e6494561d Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: X-User-Name: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: method: Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: exception: Volume dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 could not be found. Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: action_args: {'id': 'dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2'} Sep 30 10:02:03.347281 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: {{(pid=127456) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 could not be found. (HTTP 404) (Request-ID: req-0f8a63bd-b063-4a94-bab9-17923f5fdbda) Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.348389 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 30 10:02:03.349936 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 could not be found. Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 30 10:02:03.351471 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 could not be found. Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 could not be found. Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: DEBUG nova.api.openstack.wsgi [None req-934e8a96-1551-47ca-a85d-f765a6ae432f tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Returning 404 to user: Volume dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 could not be found. {{(pid=127456) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: INFO nova.api.openstack.requestlog [None req-934e8a96-1551-47ca-a85d-f765a6ae432f tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2" status: 404 len: 109 microversion: 2.1 time: 0.034491 Sep 30 10:02:03.353025 dv-cinder-master-3-iscsi devstack@n-api.service[127456]: [pid: 127456|app: 0|req: 584/1158] 10.222.0.39 () {58 vars in 1248 bytes} [Fri Sep 30 10:02:03 2022] DELETE /compute/v2.1/os-volumes/dc672b2f-cc0d-4adc-a1b4-bc6f5f7bf8d2 => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 30 10:02:03.357268 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-01f27efa-d665-4a68-a1e4-92839f85f06d tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Calling method '.version_select at 0x7f72c0ac3550>' {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 30 10:02:03.384159 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.wsgi [None req-01f27efa-d665-4a68-a1e4-92839f85f06d tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] HTTP exception thrown: Volume 1e8bbe36-4794-4485-b10c-ae79e0e23047 could not be found. Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-01f27efa-d665-4a68-a1e4-92839f85f06d tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Request method failure captured: Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: request: GET /compute/v2.1/os-volumes/1e8bbe36-4794-4485-b10c-ae79e0e23047 HTTP/1.1 Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept: application/json Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Accept-Encoding: identity Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Connection: close Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Length: 0 Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Content-Type: application/json Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Host: 10.222.0.39 Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: Openstack-System-Scope: None Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: User-Agent: python-urllib3/1.26.12 Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Auth-Token: gAAAAABjNr6bZR0iYkFHn4Pd-KuHuvDs-GYcugrhhRyCv9u7CVRlsHeO6gfZ-9RaXfArThBpZFxpRyUZuNF0ZnvUJSEdilEQL2jZ7vWyIOVScFWrsDBNNgdiWtriwbsoscO9uKjxB4vVJNX_8MrN54HHEqjDRe_DGwM1H7-K4_tuzaFXAf7_MD8 Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Id: None Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Domain-Name: None Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Identity-Status: Confirmed Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Is-Admin-Project: True Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Id: default Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Domain-Name: Default Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Project-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Role: reader,member Sep 30 10:02:03.386076 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Roles: reader,member Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/079de817fc624bd4b00f344c664d951c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/079de817fc624bd4b00f344c664d951c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.39:8080", "publicURL": "http://10.222.0.39:8080/v1/AUTH_079de817fc624bd4b00f344c664d951c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}] Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Id: 079de817fc624bd4b00f344c664d951c Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-Tenant-Name: tempest-VolumesNegativeTest-1168571359 Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Id: default Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Domain-Name: Default Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Id: 71898ee085d645bb959bef0e6494561d Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: X-User-Name: tempest-VolumesNegativeTest-1168571359-project Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: method: Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: exception: Volume 1e8bbe36-4794-4485-b10c-ae79e0e23047 could not be found. Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: action_args: {'id': '1e8bbe36-4794-4485-b10c-ae79e0e23047'} Sep 30 10:02:03.387922 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: {{(pid=127455) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 1e8bbe36-4794-4485-b10c-ae79e0e23047 could not be found. (HTTP 404) (Request-ID: req-ad45d31b-96aa-4644-95e9-9665c0c213fc) Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.389025 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 30 10:02:03.390362 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 1e8bbe36-4794-4485-b10c-ae79e0e23047 could not be found. Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 30 10:02:03.391707 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 1e8bbe36-4794-4485-b10c-ae79e0e23047 could not be found. Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 1e8bbe36-4794-4485-b10c-ae79e0e23047 could not be found. Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: ERROR nova.api.openstack.wsgi Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: DEBUG nova.api.openstack.wsgi [None req-01f27efa-d665-4a68-a1e4-92839f85f06d tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] Returning 404 to user: Volume 1e8bbe36-4794-4485-b10c-ae79e0e23047 could not be found. {{(pid=127455) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: INFO nova.api.openstack.requestlog [None req-01f27efa-d665-4a68-a1e4-92839f85f06d tempest-VolumesNegativeTest-1168571359 tempest-VolumesNegativeTest-1168571359-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/1e8bbe36-4794-4485-b10c-ae79e0e23047" status: 404 len: 109 microversion: 2.1 time: 0.034815 Sep 30 10:02:03.393082 dv-cinder-master-3-iscsi devstack@n-api.service[127455]: [pid: 127455|app: 0|req: 575/1159] 10.222.0.39 () {58 vars in 1245 bytes} [Fri Sep 30 10:02:03 2022] GET /compute/v2.1/os-volumes/1e8bbe36-4794-4485-b10c-ae79e0e23047 => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0)